What is a uppercase character

I've gathered here some not-so-obvious facts about uppercase and lowercase letters that a programmer may encounter at work. Many of you have translated strings into “all uppercase” (uppercase), “all lowercase” (lowercase), “first capital, and the rest lowercase” (titlecase). Even more popular is the case-insensitive comparison operation. On a global scale, such operations can be quite non-trivial. The post is structured as a "collection of misconceptions" with counterexamples.

1. If I convert the string to uppercase or lowercase, the number of Unicode characters does not change.

No. The text may contain lowercase ligatures, which do not correspond to one character in upper case. For example, when translating to uppercase: fi (U+FB00) -> FI (U+0046, U+0049)

2. Ligatures are a perversion, no one uses them. If they are not taken into account, then I'm right.

No. Some letters with diacritics do not have an exact match in other case, so you have to use a combined character. Let's say the Afrikaans language has the letter ʼn (U+0149). In upper case, it corresponds to a combination of two characters: (U+02BC, U+004E). If you come across a transliteration of Arabic text, you may encounter (U+1E96), which also does not have a single-character match in upper case, so you will have to replace it with (U+0048, U+0331). The Wakhi language has a letter (U+01F0) with a similar problem. You may argue that this is exotic, but there are 23,000 articles in Afrikaans on Wikipedia.

3. All right, but let's consider a combined character (involving modifying or combining code points) as one character. Then the length will still be preserved.

No. There is, for example, the letter "escet" ß (U+00DF) in German. When converted to uppercase, it turns into two SS characters (U+0053, U+0053).

4. Okay, okay, got it. We will assume that the number of Unicode characters can increase, but not more than twice.

No. There are specific Greek letters, for example, (U+0390) that turn into three Unicode characters (U+0399, U+0308, U+0301)

5. Let's talk about titlecase. Everything is simple here: I took the first character from the word, translated it into uppercase, took all the subsequent ones, translated it into lowercase.

No. Let's remember the same ligatures. If a word in lowercase begins with fl (U+FB02), then in uppercase the ligature becomes FL (U+0046, U+004C), but in titlecase it becomes Fl (U+0046, U+006C). The same with ß, but, theoretically, words cannot begin with it.

6. Those ligatures again! Well, we take the first character from the word, translate it into uppercase, if more than one character is obtained, then we leave the first one, and the rest back into lowercase. Then it will definitely work.

Won't work. There is, for example, the digraph dz (U+01F3), which can be used in text in Polish, Slovak, Macedonian or Hungarian. In uppercase it corresponds to the digraph DZ (U+01F1), and in titlecase it corresponds to the digraph Dz (U+01F2). There are other digraphs. The Greek language, on the other hand, will please you with jokes with hypogegrammen and progegrammen (fortunately, this is rarely found in modern texts). In general, the uppercase and titlecase variants for a character can be different, and there are separate entries for them in the Unicode standard.

7. Good, but at least the result of converting a character's case to uppercase or lowercase does not depend on its position in the word.

No. For example, the Greek capital sigma Σ (U+03A3) becomes a lowercase ς (U+03C2) at the end of a word and σ (U+03C3) in the middle.

8. Oh, okay, let's process the Greek sigma separately. But in any case, the same character in the same position in the text is converted in the same way.

No. For example, in most Latin languages, the lower case for I (U+0049) is i (U+0069), but not in Turkish and Azeri. There, the lower case for I is ı (U+0131) and the capital case for i is İ (U+0130). In Turkey, because of this, enchanting bugs are sometimes observed in a variety of software. And if you come across a text in Lithuanian with accents, then, for example, a capital Ì (U + 00CC), which will turn not into ì (U + 00EC), but into (U + 0069, U + 0307, ​​U + 0300) . In general, the result of the conversion also depends on the language. Most of the complex cases are described here.

9. What a horror! Well, let's now correctly convert to uppercase and lowercase. Comparing two words case-insensitively is not a problem: we translate both into lowercase and compare.

There are also many pitfalls that follow from the above. For example, it will not work with German straße and STRASSE (the former will not change, the latter will turn into strasse). There will also be problems with many of the other letters described above.

10. M-yes… Maybe then everything is in the uppercase?

And it won't always work (although much more often). But, say, if you come across the notation STRAE (yes, there is a big escet in German and Unicode too), it will not match straße. For comparisons, letters are converted according to a special Unicode table - CaseFolding, according to which both ß and SS will turn into ss.

11. A-ah-ah, this is some kind of kapets!

Here I agree.

If some symbols are not displayed for someone, write me a private message, I will replace it with a picture. not are 1:1 display or reversible. Most of these operations are for display, not normalization.

Methods casefold (), upper(), lower(), capitalize(), title(), swapcase()

str.casefold() - creates a string string, which is suitable for the case of insensitive comparisons. This is more aggressive than str.lower and may change strings that are already lowercase or cause strings to grow in length and are not intended to be displayed.

 "XßΣ".casefold() # 'xssσ' "XßΣ".lower() # 'xßς' 

The transformations that occur within casefolding are defined by the Unicode Consortium in the CaseFolding.txt file on their website.

str.upper() - takes each character in a string and converts it to its uppercase equivalent, for example:

 "This is a 'string'. ".upper() # "THIS IS A 'STRING'." 

str.lower() - does the opposite; it takes each character in a string and converts it to its lowercase equivalent:

 "This IS a 'string'.".lower() # "this is a 'string'." 

str.capitalize() - returns the capitalized version of the string, that is, it makes the first character uppercase and the rest lowercase:

 "this Is A 'String'.".capitalize() # Capitalizes the first character and lowercases all others # "This is a 'string'." 

str.title() - returns the title of the cased version of the string, i.e. each letter at the beginning of a word is made in upper case and all the rest are made in lower case:

 "this Is a 'String'".title() # "This Is A 'String'" 

str.swapcase() - str.swapcase returns a new string object with all lowercase characters swapped to uppercase and all uppercase characters swapped to lowercase:

 " this iS A STRiNG". swapcase() #Swaps case of each character # "THIS Is a strIng" 

Use as str class methods

Note that these methods can be called either on string objects (as shown above) or as a class method of str class (with an explicit call to str.upper etc.)

 str.upper("This is a 'string'") # "THIS IS A 'STRING'" 

This is especially useful when applying one of these methods to many strings at once, say map functions.

 map(str.upper,["These","are","some","'strings'"]) # ['THESE', 'ARE', 'SOME', "'STRINGS'"] 

Split a string based on delimiter into a list of strings

str.split(sep=None, maxsplit=-1)

str.split takes a string and returns a list of substrings of the original string. The behavior differs depending on whether the sep argument is provided or omitted.

If sep is not provided, or if None is not present, then splitting occurs wherever there are spaces. However, leading and trailing spaces are ignored, and multiple consecutive whitespace characters are treated the same as a single whitespace character:

 "This is a sentence.".split() # ['This', 'is', 'a', 'sentence.'] " This is a sentence. ".split() # ['This', 'is', 'a', 'sentence.'] " ".split() #[] 

sep parameter can be used to define a delimiter string. The source string is split where the separator string occurs, and the separator itself is discarded. Multiple consecutive delimiters not are treated the same way as a single one, but rather cause empty strings to be created.

 "This is a sentence.".split('') # ['This', 'is', 'a', 'sentence.'] "Earth,Stars,Sun,Moon".split(',') # ['Earth', 'Stars', 'Sun', 'Moon'] " This is a sentence. ".split('') # ['', 'This', 'is', '', '', '', 'a', 'sentence.', '', ''] "This is a sentence.".split('e') # ['This is a s', 'nt', 'nc', '.'] "This is a sentence.".split('en') # ['This is a s', 't', 'ce.'] 

The default is to split by every occurrence of a delimiter, however the maxsplit parameter limits the number of splits that occur. The default value -1 means there is no limit:

 "This is a sentence.".split('e', maxsplit=0) # ['This is a sentence.'] "This is a sentence.".split('e', maxsplit=1) # ['This is a s', 'ntence.'] "This is a sentence.".split('e', maxsplit=2) # ['This is a s', 'nt', 'nce.'] "This is a sentence.".split('e', maxsplit=-1) # ['This is a s', 'nt', 'nc', '.'] 

str.rsplit(sep=None, maxsplit=-1)

str.rsplit ("right split") is different from str.split ("left split") when maxsplit is specified. Splitting starts at the end of the string, not at the beginning:

 "This is a sentence.".rsplit('e', maxsplit=1) # ['This is a sentenc', '.'] "This is a sentence.".rsplit('e', maxsplit=2) # ['This is a sent', 'nc', '.'] 

Note: Python defines a maximum number of splits are performed, while most other programming languages ​​specify a maximum number of substrings created. This can create confusion when porting or comparing code.

Replace all occurrences of one substring with another substring

Python's str type also has a method for replacing occurrences of one substring with another substring in a given string. For more complex cases, re.sub can be used. str.replace(old, new[ count]) :

str.replace takes two arguments, old and new , containing the old substring to be replaced with the new substring. The optional argument count specifies the number of replacements to be:

For example, in order to replace 'foo' with 'spam' in the next line, we can call str.replace with old = 'foo ' and new = 'spam' :

 "Make sure to foo your sentence.".replace('foo', 'spam') # "Make sure to spam your sentence." 

If the given string contains multiple examples that match old the argument, all occurrences are replaced by the value supplied in new :

 "It can foo multiple examples of foo if you want. ".replace('foo', 'spam') # "It can spam multiple examples of spam if you want." 

unless of course we supply a value for count . In this case, count entries are going to be replaced:

 """It can foo multiple examples of foo if you want, \ or you can limit the foo with the third argument.""".replace('foo', 'spam', 1) # 'It can spam multiple examples of foo if you want, or you can limit the foo with the third argument.' 

str.format and f-strings: format values ​​to string

Python provides string interpolation and formatting functionality via str.format function introduced in version 2.6 and F-strings introduced in version 3.6.

The following variables are given:

 i = 10 f = 1.5 s = "foo" l = ['a', 1, 2] d = {'a': 1, 2: 'foo'} 

Let's see different formatting of the string

 "{} {} {} {} {}". format(i, f, s, l, d) str.format("{} {} {} {} {}", i, f, s, l, d) "{0} {1} {2} {3} {4}". format(i, f, s, l, d) "{0:d} {1:0.1f} {2} {3!r} {4!r}".format(i, f, s, l, d) "{i:d} {f:0.1f} {s} {l!r} {d!r}".format(i=i, f=f, s=s, l=l, d=d) 

All statements above are equivalent "10 1.5 foo ['a', 1, 2] {'a': 1, 2: 'foo'}"

 f"{i} {f} {s} { l}{d}" f"{i:d} {f:0.1f} {s} {l!r} {d!r}" 

For reference, Python also supports C-style classifiers for string formatting. The examples below are equivalent to those above, but the str.format variants are preferred due to advantages in flexibility, notation consistency, and extensibility:

 "%d %0.1f %s %r %r" % (i, f, s, l, d) "%(i)d %(f)0.1f %(s)s %(l)r %(d)r" % dict(i=i, f=f, s=s, l=l, d=d ) 

Parentheses are used for interpolation in str.format can also be numbered to reduce duplication when formatting strings. For example, the following is equivalent to:

 "I am from {}. I love cupcakes from {}!".format("Australia", "Australia") #"I am from Australia. I love cupcakes from Australia!" "I am from {0}. I love cupcakes from {0}!".format("Australia") #"I am from Australia. I love cupcakes from Australia!" 

While the official python documentation is thorough as usual, pyformat.info has a large set of examples with detailed explanations.

In addition, { and } characters can be escaped using double brackets:

 "{{'{}': {}, '{}': {}}}".format("a" , 5, "b", 6) # "{'a': 5, 'b': 6}" 

See Format string for more information. str.format() was proposed in PEP 3101 and F-strings in PEP 498 .

Counting the number of occurrences of a substring in a string

One method is available for counting the number of occurrences of a substring in another string, str.count . str.count(sub[ start[ end]])

str.count returns int indicating the number of non-overlapping occurrences of substrings sub in another string. The optional arguments start and end indicate the start and end at which the search will occur. Default start = 0 and end = len(str) means the entire string will be searched:

 s = "She sells seashells by the seashore." s.count("sh") #2 s.count("se") #3 s.count("sea") #2 s.count("seashells") # 1 

By giving different values ​​for start , end , we can get a more localized lookup and count, for example if start is equal to 13 call to:

 s.count("sea", start) # 1 

is equivalent to:

 t = s[start:] t.count("sea") # 1 

Check the start and end characters of a string

To check the start and end of a given string in Python, you can use the str.startswith() and str.endswith() methods. str.startswith(prefix[ start[ end]])

As the name implies, str.startswith is used to check if the given string starts with the given characters in prefix .

 s = "This is a test string" s.startswith("T") # True s.startswith("Thi") # True s.startswith("thi") # False 

The optional arguments start and end specify the start and end points from which testing will start and end. In the following example, by giving a start value of 2 our string will be looked up from position 2 and then:

 s.startswith("is", 2) # True 

This gives True since s[2] == 'i' and s[3] == 's' .

You can also use tuple to check if it starts with any of the stringset

 s.startswith(('This', 'That')) # True s.startswith(('ab', 'bc')) # False 

str.endswith(prefix[ start[ end]]) - exactly like str.startswith with the only difference that it looks for ending characters and not starting characters. For example, to check if a string ends with a full stop, you can write:

 s = "this ends in a full stop. " s.endswith('.') # True s.endswith('!') # False 

as with startswith more than one character can be used to end a sequence:

 s.endswith('stop.') # True s.endswith('Stop.') # False 

You can also use tuple to check if it ends with any of the stringset

 s.endswith(('.', 'something')) # True s.endswith(('ab', 'bc')) # False 

Check what string 9 consists ofThe 0054

Python str type also has a number of methods that can be used to evaluate the contents of a string. These are str.isalpha , str.isdigit , str.isalnum , str.isspace . The capitalization can be checked with str.isupper , str.islower and str.islower and str.isalnum str.isalpha

str.isalpha takes no arguments and returns True if all characters in the given string are alphabetic, for example:

 "Hello World". isalpha() # contains a space # False "Hello2World".isalpha() # contains a number # False "HelloWorld!".isalpha() # contains punctuation # False "HelloWorld".isalpha() # True 

In the edge case, an empty string evaluates to False when using "".isalpha() . str.isupper , str.islower , str.istitle

These methods check for the use of capital letters in a given string.

str.isupper is a method that returns True if all characters in the given string are uppercase and False otherwise.

 "HeLLO WORLD".isupper() # False "HELLO WORLD".isupper() # True "".isupper() # False 

On the other hand, str.islower is a method that returns True if all characters in the given string are lowercase and False otherwise.

 "Hello world".islower() # False "hello world".islower() # True "".islower() # False 

str. istitle returns True if the given title string is cased; that is, each word begins with an uppercase letter followed by lowercase letters.

 "hello world".istitle() # False "Hello world".istitle() # False "Hello World".istitle() # True "".istitle() False 

Methods str.isdecimal , str.isdigit , str.isnumeric

str.isdecimal returns a string whether a sequence of decimal digits is suitable for representing a decimal number.

str.isdigit includes a digit not in a form suitable for representing a decimal number, such as superscript digits.

str.isnumeric includes any numeric value, even if not numeric, such as values ​​outside the range 0-9.

 isdecimal isdigit isnumeric 12345 True True True ១2҃໔5 True True True ①²³🄅₅ False True True ⑩⒓ False False True Five False False False 

Byte strings ( bytes in Python 3, str in Python 2), only supports isdigit , which only checks for basic ASCII digits.

Like str.isalpha an empty string evaluates to False . str.isalnum

This is a combination of str.isalpha and str.isnumeric , specifically it has the value True if all characters in the given string are literal - numeric, i.e. they consist of alphabetic or numeric characters:

 "Hello2World".isalnum() # True "HelloWorld".isalnum() # True "2022".isalnum() # True "Hello World".isalnum() # contains whitespace # False 

str.isspace - Returns True if the string contains only whitespace characters.

 "\t\r\n".isspace() # True " ".isspace() # True 

Sometimes a string looks "empty" but we don't know if it is because it contains only spaces or no characters at all

 "".isspace() # False 

To cover this case we need an extra test

 my_str = '' my_str. isspace() # False my_str.isspace() or not my_str # True 

But the shortest way to check if a string is empty or contains only whitespace characters is to use strip (with no arguments it strips all leading and trailing whitespace characters)

 not my_str.strip() # True 

str.translate: translate characters in string

Python supports a translate method on the str type, which allows you to specify a translation table (used for replacement), as well as any characters to be removed in the process.

str.translate(table[ deletechars]) - parameter table is a lookup table that defines the mapping from one character to another. deletechars is a list of characters to be removed from the string.

9The 0048 maketrans method ( str.maketrans in Python 3 and string.maketrans in Python 2) allows you to create a translation table.

 translation_table = str.maketrans("aeiou", "12345") my_string = "This is a string!" translated = my_string.translate(translation_table) # 'Th4s 3s 1 str3ng!' The 

translate method returns a string that is a translated copy of the original string. You can set table argument None if only required to remove characters.

 'this syntax is very useful'.translate(None, 'aeiou') 'ths syntx s vry sfl' 

Remove unwanted leading/trailing characters from a string

Three methods provided that offer the ability to strip leading and trailing characters from a string: str.strip , str.rstrip and str. lstrip All three methods have the same signature, and all three return a new string object with unwanted characters removed. str.strip([chars])

str.strip acts on the given string and removes (strips) or any leading trailing characters contained in the argument chars ; if chars is not included or there is no None , all whitespace characters are removed by default. For example:

 " a line with leading and trailing space ".strip() # 'a line with leading and trailing space' 

If chars are supplied, all characters contained in it are removed from the string that is returned. For example:

 ">>> a Python prompt".strip('>') # strips the '>' character and the space after it #'a Python prompt' 

str.rstrip([chars]) and str.lstrip([chars]) - These methods have the same semantics and arguments as str.strip() , their difference lies in the direction they come from begin. str.rstrip() starts from the end of the string while str.lstrip() splits from the beginning of the string.

For example, when using str.rstrip :

 " spacious string ".rstrip() # ' spacious string' 

While using str.lstrip :

 " spacious string ".rstrip() # 'spacious string' " spacious string ".rstrip(). lstrip() # 'spacious string' 

Case-insensitive string comparison

Case-insensitive string comparison seems trivial, but it's not. This section only covers Unicode strings (the default in Python 3). Note that Python 2 may have minor drawbacks compared to Python 3 - the later handling of unicode is much more complete.

The first thing to note is that unicode case-removal conversions are not trivial. There is a text for which text.lower() != text.upper().lower() , For example, "ß" :

 >>> "ß".lower() 'ß' >>> "ß".upper().lower() 'ss' 

But suppose you wanted to case insensitively compare "BUSSE" and "Buße" . Heck, you probably also want to compare "BUSSE" and "BUẞE" equal - this is a new form of capital. The recommended way is to use casefold :

 help(str.casefold) """ Help on method_descriptor: casefold(self, /) Return a version of the string suitable for caseless comparisons """ 

Don't just use lower . If casefold isn't available, doing .upper().lower() helps (but only a little).

Then you should consider accents. If the font renderer is fine, you probably think "ê" == "ê" - but it's not true :

 "ê" == "ê" # False 

This is because they are actually

 unicodedata [unicodedata.name(char) for char in "ê"] # ['LATIN SMALL LETTER E WITH CIRCUMFLEX'] [unicodedata.name(char) for char in "ê"] # ['LATIN SMALL LETTER E', 'COMBINING CIRCUMFLEX ACCENT' 

The easiest way to deal with this is unicodedata.normalize . You probably want to use NFKD normalization, but feel free to check the documentation. Then one

 unicodedata.normalize("NFKD", "ê") == unicodedata.normalize("NFKD", "ê") # True 

To finish, this is expressed in functions here:

 import unicodedata def normalize_caseless(text): return unicodedata.normalize("NFKD", text.casefold()) def caseless_equal(left, right): return normalize_caseless(left) == normalize_caseless(right) 

Concatenate a list of strings into one string

A string can be used as a delimiter to join a list of strings together into one string with join() method. For example, you can create a string where each element in the list is separated by a space.

 " ".join(["once","upon","a","time"]) # "once upon a time" 

In the following example, string elements are separated by three hyphens.

 "---".join(["once", "upon", "a", "time"]) # "once---upon---a---time" 

Useful string module constants

Python string The Python module provides constants for operations involving strings. To use them, import string module:

 import string 

Combination of ascii_lowercase and ascii_uppercase :

 string.ascii_letters # 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ' 

string.ascii_lowercase :

Contains all ASCII lowercase characters:

 string.ascii_lowercase # 'abcdefghijklmnopqrstuvwxyz' 

string.ascii_uppercase :

Contains all uppercase ASCII characters:

 string. ascii_uppercase # 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 

string.digits - contains all decimal digits:

 string.digits # '0123456789' 

string.hexdigits - contains all hex characters:

 string.hexdigits # '0123456789abcdefABCDEF' 

string.octaldigits - contains all octal characters:

 string.octaldigits # '01234567' 

string.punctuation - contains all characters that are considered punctuation in 9_`{|}~'

string.whitespace - contains all ASCII characters that are considered whitespace:

 string.whitespace # ' \t\n\r\x0b\x0c' 

In script mode, print(string.whitespace) will print the actual characters, use str to get the above string returned.

string.printable - contains all characters that are considered printable; combination of string.digits , string.ascii_letters 9_`{|}~ \t\n\r\x0b\x0c'

Reversing a string (reverse)

A string can be reversed using the built-in function reversed() , which takes a string and returns an iterator in reverse order.

 reversed('hello') #  [char for char in reversed('hello')] # ['o', 'l', 'l', 'e', ​​'h'] 

reversed() can be wrapped in a ''.join() call to make a string from an iterator.

 ''.join(reversed('hello')) # 'olleh' 

While using reversed() may be more readable for uninitiated Python users, using extended slicing in increments of -1 is faster and more concise. Here, try implementing this as a function:

 def reversed_string(main_string): return main_string[::-1] reversed_string('hello') # 'olleh' 

Line alignment

Python provides functions for line alignment, allowing you to pad text to make it easier to align different lines.

Here is an example of str.ljust and str.rjust :

 interstates_lengths = { 5: (1381, 2222), 19:(63, 102), 40: (2555, 4112), 93: (189.305) } for road, length in interstates_lengths. items(): miles,kms = length print('{} -> {} mi.({} km.)'.format(str(road).rjust(4), str(miles).ljust(4), str(kms).ljust(4) )) # 5 -> 1381 mi. (2222 km.) # 19 -> 63 mi. (102 km.) # 40 -> 2555 mi. (4112 km.) # 93 -> 189 mi. (305 km.) 

ljust and rjust are very similar. Both have a width parameter and an optional fillchar parameter. Any string generated by these functions is at least as long as the width parameter that was passed into the function. If the string is longer than width alread, it is not truncated. fillchar argument that defaults to a space character ' ' must be a single character, not a multicharacter string.

ljust end-of-line pad function it is called on with fillchar until width character length. rjust function to pad the beginning of a line in a similar manner. Thus, l and r in the names of these functions refer to the side that the source string, and not fillchar , is located in the output string.

Convert between str or byte data and unicode characters

The contents of files and network messages can be encoded characters. They often need to be converted to unicode to display correctly.

In Python 3, you may need to convert byte arrays (called "byte literal") to Unicode character strings. The default is now a Unicode string, and byte string literals must now be entered as b'' , b"" , etc. Byte literal will return True in isinstance(some_val, byte) , assuming some_val to be a string that can be encoded in bytes.

 # You get from file or network "© abc" encoded in UTF-8 s = b'\xc2\xa9 abc' # s is a byte array, not characters # In Python 3, the default string literal is Unicode; byte array literals need a leading b s[0] # b'\xc2' - meaningless byte (without context such as an encoding) type(s) # bytes - now that byte arrays are explicit, Python can show that. 
Learn more

		
Creative hopscotch ideas
				
Fairy tales short version
				
Teaching long a sound
				
Ing words for kids
				
Feelings activities children
				
One two buckle shoe
				
Tongue twister about math
				
Humpty dumpty fairy tale
				
Silly rhyming words
				
Show me bugs
				
Robot abc song