Python Details

This blog reveals some details of vital importance in python coding.

Sort() + lambda expression

Lambda expression can be used to be the parameter ‘cmp’ of sort() to determine the way of sort.

Default(Ascending order):

1	>>> list.sort() = list.sort(cmp = lambda x, y: cmp(x, y))

Descending order:

1	>>> list.sort(cmp = lambda x, y: cmp(y, x))

e.g. Given a list a = [‘1’, ‘9’, ‘3’, ‘5’, ‘7’], converge list element and return the biggest one.
Solution as follows:

1
2
3

>>> a.sort(cmp = lambda x, y: cmp(y+x, x+y))
>>> a
['9', '7', '5', '3', '1']

Zip

Zip function is used to merge two single elements into a (key, value) tuple.
e.g.

>>> list_keys = ['Country', 'Total']
>>> list_values = [['US', 'SU'], [118, 73]]
>>> zip(list_keys, list_values)
[('Country', ['US', 'SU']), ('Total', [118, 73])]

Convert (key, value) pairs to dict:

1 2	>>> dict(list(zip(list_keys, list_values))) >>> {'Country': ['US', 'SU'], 'Total': [118, 73]}

Generator:

Generator will not create complete list in order to save space.
e.g. Create a generator:

1
2
3

>>> g = (x * x for x in range(10))
>>> g
<generator object <genexpr> at 0x104feab40>

Visit elements:

>>> g.next()
0
>>> g.next()
1
>>> g.next()
4

Iteration:

>>> for n in g:
...     print n
...
0
1
4

yield is used to return a value into generator.
Convert function into generator:

>>> def generator():
...     for i in range(3):
...         yield i * i
...
>>> g = generator()
>>> g
<generator object generator at 0x10a1671e0>
>>> for i in g:
>>> ...     print i
>>> ...
>>> 0
>>> 1
>>> 4

Conditional operator

Condition operator has many forms in python:
If-else:

1
2
3

>>> a = 1 if 1 > 0 else 0
>>> a
1

List/tuple:

1
2
3

>>> a = (1, 0)[1 > 0]
>>> a
0

Orr:

1
2
3

>>> a = [1, 0][1 > 0]
>>> a
0

Notice that True/False represents 1/0 when they are index.

List vs np.array

List can contain multiple elements from different categories. But np.array can only contain homogeneous elements.

So for lists we have:

1
2
3

>>> list = [1, 'z', True]
>>> list
[1, 'z', True]

But for np.array:

1 2	>>> np.array([1, 'z', True]) array(['1', 'z', 'True'], dtype='\|S21')

It turns out different categorical elements will be transformed into one specific category. In here, they are transformed to string.

Nested function

The nested function is a very useful tool to achieve some goals. For instance, suppose we have a function that $ y = x mod 2 + 5$, the mission is to pass three x to function and return y. If we are not allowed to use flexible arguments(mention later), how can we do it? The answer is calculating them respectively.

def mod2plus5(x1, x2, x3):
    y1 = x1 % 2 + 5
    y2 = x2 % 2 + 5
    y3 = x3 % 2 + 5
    return (y1, y2, y3)
>>> mod2plus5(4, 15, 25)
(5, 6, 6)

Although we have achieved our goal, it is not concisely at all, right? So, that’s the chance to introduce nested function. Nested function is a special function and we can call it as inner function. Intuitively, we can understand it as a subfunction within a parent function. Anyway, the second way of completing our mission is to use it.

def mod2plus5(x1, x2, x3):
    def inner(x):
        return x % 2 + 5
    return (inner(x1), inner(x2), inner(x3))
>>> mod2plus5(4, 15, 25)
(5, 6, 6)

As you can see, we achieved our goal elegantly and efficiently, right? Actually, there is a third way to achieve it called flexible arguments, similarly efficient at all. We will discuss it later.

You may have a question that if I have two variables with same name in different scope of customized functions, how does python intepreter recognize and use them? So, let us discuss python scope.

Take a look at this code fragment.

def outer():
    n = 1
    def inner():
        n = 2
        print(n)
    inner()
    print(n)

What are the outputs? Interesting question! Right? The key point of this problem are the second n and the second ‘print’. In fact, the output is(can be verified by yourself):

1
2
3

>>> outer()
2
1

As you can see, two ‘print’ keywords print their own variable n in their scope respectively. In other words, the variable n in inner scope makes no influence on enclosing scope. So, another question is, if we want inner variables can be detected outside, what should we do?

The answer is quite simple – using nonlocal keyword.

def outer():
    n = 1
    def inner():
    	nonlocal n
        n = 2
        print(n)
    inner()
    print(n)

Let’s check our output:

1
2
3

>>> outer()
2
2

To summarize, python’s name references search at most 4 scopes, the local scope, those of enclosing functions(if any), global, built-in. That’s called LEGB formula. Built-in scope means python’s built-in repository, such as ‘print’, ‘def’ and so on.

Regular Expression

Regular expression is a strong weapon to match Strings. For example, we want to match E-mail address.

Fixed length matching

\d matches single number
\w matches single letter or number

e.g.

‘00\d’ matches ‘007’
‘\w\w\d’ matches ‘py3’

Variable length matching

*matches any number chars(include 0)
+matches at least one char
?matches 0 or 1 char
{n}matches n chars
{n, m}matches n-m chars

e.g.

‘py.’ matches ‘pyc’, ‘pyo’, ‘py!’ etc
\d{3}\s+\d{3,8} matches ‘010 82623372’
\d{3}-\d{3,8} matches ‘010-12345’

Advanced

要做更精确地匹配，可以用[]表示范围，比如：

[0-9a-zA-Z\_]可以匹配一个数字、字母或者下划线；
[0-9a-zA-Z\_]+可以匹配至少由一个数字、字母或者下划线组成的字符串，比如'a100'，'0_Z'，'Py3000'等等；
[a-zA-Z\_][0-9a-zA-Z\_]*可以匹配由字母或下划线开头，后接任意个由一个数字、字母或者下划线组成的字符串，也就是Python合法的变量；
[a-zA-Z\_][0-9a-zA-Z\_]{0, 19}更精确地限制了变量的长度是1-20个字符（前面1个字符+后面最多19个字符）。

A|B可以匹配A或B，所以(P|p)ython可以匹配'Python'或者'python'。

^表示行的开头，^\d表示必须以数字开头。

$表示行的结束，\d$表示必须以数字结束。

re module

有了准备知识，我们就可以在Python中使用正则表达式了。Python提供re模块，包含所有正则表达式的功能。由于Python的字符串本身也用\转义，所以要特别注意：

1
2
3

s = 'ABC\\-001' # Python的字符串
# 对应的正则表达式字符串变成：
# 'ABC\-001'

因此我们强烈建议使用Python的r前缀，就不用考虑转义的问题了：

1
2
3

s = r'ABC\-001' # Python的字符串
# 对应的正则表达式字符串不变：
# 'ABC\-001'

先看看如何判断正则表达式是否匹配：

>>> import re
>>> re.match(r'^\d{3}\-\d{3,8}$', '010-12345')
<_sre.SRE_Match object at 0x1026e18b8>
>>> re.match(r'^\d{3}\-\d{3,8}$', '010 12345')
>>>

match()方法判断是否匹配，如果匹配成功，返回一个Match对象，否则返回None。常见的判断方法就是：

test = '用户输入的字符串'
if re.match(r'正则表达式', test):
    print 'ok'
else:
    print 'failed'

Split Strings

用正则表达式切分字符串比用固定的字符更灵活，请看正常的切分代码：

1 2	>>> 'a b c'.split(' ') ['a', 'b', '', '', 'c']

嗯，无法识别连续的空格，用正则表达式试试：

1 2	>>> re.split(r'\s+', 'a b c') ['a', 'b', 'c']

无论多少个空格都可以正常分割。加入,试试：

1 2	>>> re.split(r'[\s\,]+', 'a,b, c d') ['a', 'b', 'c', 'd']

再加入;试试：

1 2	>>> re.split(r'[\s\,\;]+', 'a,b;; c d') ['a', 'b', 'c', 'd']

Group

除了简单地判断是否匹配之外，正则表达式还有提取子串的强大功能。用()表示的就是要提取的分组（Group）。比如：

^(\d{3})-(\d{3,8})$分别定义了两个组，可以直接从匹配的字符串中提取出区号和本地号码：

>>> m = re.match(r'^(\d{3})-(\d{3,8})$', '010-12345')
>>> m
<_sre.SRE_Match object at 0x1026fb3e8>
>>> m.group(0)
'010-12345'
>>> m.group(1)
'010'
>>> m.group(2)
'12345'

如果正则表达式中定义了组，就可以在Match对象上用group()方法提取出子串来。

注意到group(0)永远是原始字符串，group(1)、group(2)……表示第1、2、……个子串。

Compile

当我们在Python中使用正则表达式时，re模块内部会干两件事情：

编译正则表达式，如果正则表达式的字符串本身不合法，会报错；
用编译后的正则表达式去匹配字符串。

如果一个正则表达式要重复使用几千次，出于效率的考虑，我们可以预编译该正则表达式，接下来重复使用时就不需要编译这个步骤了，直接匹配：

>>> import re
# 编译:
>>> re_telephone = re.compile(r'^(\d{3})-(\d{3,8})$')
# 使用：
>>> re_telephone.match('010-12345').groups()
('010', '12345')
>>> re_telephone.match('010-8086').groups()
('010', '8086')

编译后生成Regular Expression对象，由于该对象自己包含了正则表达式，所以调用对应的方法时不用给出正则字符串。