Stackoverflow: So you want to write Pythonic code in Python
Advance python and some sweet python tips and tricks.
- Lambda and Comphresions
- Decorators
- Context Managers
- Iterators and Generators
- Functools and Itertools (This is going to blow your mind)
- Some Python Tricks
- Refer these To be GREAT IN PYTHON
Functions in python are first class Objects that means you can assign them to variable, store them in data structure, pass them as a parameter to other functions and even return them from other function
# addition function
def add(x, y):
return x+y
print (f" function is add(5,6) = {add(5,6)}")
# you can assign them to other variable
myAdd = add
# wait you can also delete the add function and the myAdd still points to underlying function
del add
print (f" function is myAdd(5,6) = {myAdd(5,6)}")
#functions have their own set of attributes
print(f"{myAdd.__name__}")
# to see a complete list of attributes of a function type dir(myAdd) in console
# functions as data structures
List_Funcs = [str.upper , str.lower , str.title]
for f in List_Funcs:
print (f , f("aI6-saturdays"))
So lambdas are a sweet Little anonymous Single-Expression functions
add_lambda = lambda x , y : x+y # lambda automaticaly returns the value after colon
print(f"lambda value add_lambda(2,3)= {add_lambda(2,3)}") #you call lambda function as normal functions
You :- But Wait it's an anonymous function and how can you give it a name
StackOverflow :- Relax, Searching for another example
def someFunc(func):
quote = func("We will democratize AI ")
return quote
# here the lambda function is passes to a normal function
# the lambda here is anonymous and the parameter my_sentence = We will democratize AI
# so we are adding some text of ours and returning the string
someFunc(lambda my_sentence: my_sentence+"by teaching everyone AI")
# here is one more example
tuples = [(1, 'd'), (2, 'b'), (4, 'a'), (3, 'c')]
sorted(tuples, key=lambda x: x[1])
# list comphrensions
l = [ x for x in range(20)]
even_list = [x for x in l if x%2==0]
even_list_with_Zero = [x if x%2==0 else 0 for x in l ]
l , even_list ,even_list_with_Zero
# dictionary comphrension
d = {x: x**2 for x in range(2,6)}
flip_key_value = {value:key for key,value in d.items()}
d , flip_key_value
Decorators
Python’s decorators allow you to extend and modify the behavior of a callable (functions, methods, and classes) without permanently modifying the callable itself.
Any sufficiently generic functionality you can tack on to an existing class or function’s behavior makes a great use case for decoration. This includes the following:
- logging
- enforcing access control and authentication
- instrumentation and timing functions
- rate-limiting
- caching, and more
Imagine that you some 50 functions in your code. Now that all functions are working you being a great programmer thought of optimizing each function by checking the amount of time it takes and also you need to log the input/output of few functions. what are you gonna do ?
Without decorators you might be spending the next three days modifying each of those 50 functions and clutter them up with your manual logging calls. Fun times, right?
def my_decorator(func):
return func # It's simple right it takes a function as it's parameter and returns it
def someFunc():
return "Deep learning is fun"
someFunc = my_decorator(someFunc) # it is similar to i = i + 1
print(f" someFunc value = {someFunc()}")
# now just to add syntatic sugar to the code so that we can brag how easy and terse python code is
# we gonna write this way
def my_decorator(func):
return func
@my_decorator # the awesomeness of this block of code lies here which can be used as toggle switch
def someFunc():
return "Deep learning is fun"
print(f" someFunc value = {someFunc()}")
Stackoverflow :- Now that you got a little taste of Decorators let's write another decorator that actually does something and modifies the behavior of the decorated function.
# This blocks contains and actual implementation of decorator
import time
#import functools
def myTimeItDeco(func):
#@functools.wraps(func)
def wrapper(*args,**kwargs):
starttime = time.time()
call_of_func = func(*args,**kwargs) # this works because you can function can be nested and they remember the state
function_modification = call_of_func.upper()
endtime = time.time()
return f" Executed output is {function_modification} and time is {endtime-starttime} "
return wrapper
@myTimeItDeco
def myFunc(arg1,arg2,arg3): # some arguments of no use to show how to pass them in code
"""Documentation of a obfuscate function"""
time.sleep(2) # just to show some complex calculation
return "You had me at Hello world"
myFunc(1,2,3) , myFunc.__doc__ , myFunc.__name__
You :- Why didn't I got the doc and the name of my function. Hmmm....
StackOverflow :- Great Programmers use me as there debugging tool, so use It.
Hints functools.wrap
StackOverflow : - Applying Multiple Decorators to a Function (This is really fascinating as it's gonna confuse you)
def strong(func):
def wrapper():
return '<strong>' + func() + '</strong>'
return wrapper
def emphasis(func):
def wrapper():
return '<em>' + func() + '</em>'
return wrapper
@strong
@emphasis
def greet():
return 'Hello!'
greet()
# this is your assignment to understand it hints strong(emphasis(greet))()
#Disclaimer Execute this at your own risk
#Only 80's kids will remember this
import dis
dis.dis(greet)
# let's open a file and write some thing into it
file = open('hello.txt', 'w')
try:
file.write('Some thing')
finally:
file.close()
You :- ok now that I have wrote something into the file I want to read it, but try and finally again, it suck's. There should be some other way around
StackOverflow :- Context manger for your Rescue
with open("hello.txt") as file:
print(file.read())
You :- That's pretty easy but what is with
Stackoverflow :- It helps python programmers like you to simplify some common resource management patterns by abstracting their functionality and allowing them to be factored out and reused.
So in this case you don't have to open and close file all done for you automatically.
# A good pattern for with use case is this
some_lock = threading.Lock()
# Harmful:
some_lock.acquire()
try:
# Do something complicated because you are coder and you love to do so ...
finally:
some_lock.release()
# Better :
with some_lock:
# Do something awesome Because you are a Data Scientist...
You :- But I want to use with for my own use case how do I do it?
StackOverflow :- Use Data Models and relax
# Python is language full of hooks and protocol
# Here MyContextManger abides context manager protocol
class MyContextManger:
def __init__(self, name):
self.name=name
# with statement automatically calls __enter__ and __exit__ methods
def __enter__(self): ## Acquire the lock do the processing in this method
self.f = open(self.name,"r")
return self.f
def __exit__(self,exc_type,exc_val,exc_tb): ## release the lock and free allocated resources in this method
if self.f:
self.f.close()
with MyContextManger("hello.txt") as f:
print(f.read())
You :- It works but what are those parameters in exit method
Stackoverflow :- Google It !
You :- But writing a class in python is hectic, I want to do functional Programming
StackOverflow :- Use Decorators
from contextlib import contextmanager
@contextmanager
def mySimpleContextManager(name):
try:
f = open(name, 'r')
yield f
finally:
f.close()
with mySimpleContextManager("hello.txt") as f:
print(f.read())
You :- Ok, that's what I call a pythonic code but what is yeild
Stackoverflow :- Hang On!
Iterators and Generators
An iterator is an object representing a stream of data; this object returns the data one element at a time. A Python iterator must support a method called next() that takes no arguments and always returns the next element of the stream. If there are no more elements in the stream, next() must raise the StopIteration exception. Iterators don’t have to be finite, though; it’s perfectly reasonable to write an iterator that produces an infinite stream of data.
The built-in iter() function takes an arbitrary object and tries to return an iterator that will return the object’s contents or elements, raising TypeError if the object doesn’t support iteration. Several of Python’s built-in data types support iteration, the most common being lists and dictionaries. An object is called iterable if you can get an iterator for it.
l = [1,2,3]
it = l.__iter__() ## same as iter(l)
it.__next__() ## gives 1
next(it) ## gives 2
next(it) ## gives 3
next(it) ## gives error StopIteration
#lets replicate the simple range method
class MyRange:
def __init__(self,start,stop):
self.start = start -1
self.stop = stop
def __iter__(self):
return self
def __next__(self):
self.start = self.start+1
if self.start<self.stop:
return self.start
else:
raise StopIteration()
for i in MyRange(2,10):
print(i)
You :- Again a class
StackOverflow :- OK here's a easy way Use Generators
They Simplify writing Iterators, kind of iterable you can only iterate over once. Generators do not store the values in memory, they generate the values on the fly so no storage is required. So you ask one value it will generate and spit it out
def myRange(start,stop):
while True:
if start<stop:
yield start
start = start+1
else:
return
for i in myRange(2,10):
print(i)
You’re doubtless familiar with how regular function calls work in Python or C. When you call a function, it gets a private namespace where its local variables are created. When the function reaches a return statement, the local variables are destroyed and the value is returned to the caller. A later call to the same function creates a new private namespace and a fresh set of local variables. But, what if the local variables weren’t thrown away on exiting a function? What if you could later resume the function where it left off? This is what generators provide; they can be thought of as resumable functions.
Any function containing a yield keyword is a generator function; this is detected by Python’s bytecode compiler which compiles the function specially as a result.
When you call a generator function, it doesn’t return a single value; instead it returns a generator object that supports the iterator protocol. On executing the yield expression, the generator outputs the value start , similar to a return statement. The big difference between yield and a return statement is that on reaching a yield the generator’s state of execution is suspended and local variables are preserved. On the next call to the generator’s next() method, the function will resume executing.
# generator comphresnsive
l = ( x for x in range(20))
l
[*l]
Let’s look in more detail at built-in functions often used with iterators.
Two of Python’s built-in functions, map() and filter() duplicate the features of generator expressions:
map(f, iterA, iterB, ...) returns an iterator over the sequence
f(iterA[0], iterB[0]), f(iterA[1], iterB[1]), f(iterA[2], iterB[2]), ....
filter(predicate, iter) returns an iterator over all the sequence elements that meet a certain condition, and is similarly duplicated by list comprehensions. A predicate is a function that returns the truth value of some condition; for use with filter(), the predicate must take a single value.
# why did it returned an empty list think?
[*map(lambda x :x **2 , l)]
[*filter(lambda x :x%2!=0,l)]
zip(iterA, iterB, ...) takes one element from each iterable and returns them in a tuple:
z = zip(['a', 'b', 'c'], (1, 2, 3))
for x , y in z:
print (x,y)
These two python modules are super helpful in writing Efficient Functional Code
# reduce
from functools import reduce
l = (x for x in range(1,10))
reduce(lambda x,y : x+y , l)
For programs written in a functional style, you’ll sometimes want to construct variants of existing functions that have some of the parameters filled in. Consider a Python function f(a, b, c); you may wish to create a new function g(b, c) that’s equivalent to f(1, b, c); you’re filling in a value for one of f()’s parameters. This is called “partial function application”.
The constructor for partial() takes the arguments (function, arg1, arg2, ..., kwarg1=value1, kwarg2=value2). The resulting object is callable, so you can just call it to invoke function with the filled-in arguments.
from functools import partial
def log(message, subsystem):
"""Write the contents of 'message' to the specified subsystem."""
print('%s: %s' % (subsystem, message))
...
server_log = partial(log, subsystem='server')
server_log('Unable to open socket')
from itertools import islice ,takewhile,dropwhile
# here is a very simple implementation of the fibonacci sequence
def fib(x=0 , y=1):
while True:
yield x
x , y = y , x+y
list(islice(fib(),10))
list(takewhile(lambda x : x < 5 , islice(fib(),10)))
list(dropwhile(lambda x : x < 5 , islice(fib(),10)))
list(dropwhile(lambda x :x<11 , takewhile(lambda x : x < 211 , islice(fib(),15))))
To read more about itertools https://docs.python.org/3.6/howto/functional.html#creating-new-iterators
#normal calculator prorgramm
def calculator(operator , x , y):
if operator=="add":
return x+y
elif operator=="sub":
return x-y
elif operator == "div":
return x/y
elif operator=="mul":
return x*y
else :
return "unknow"
calculator("add",2,3)
#Pythonic way
calculatorDict = {
"add":lambda x,y:x+y,
"sub":lambda x,y:x-y,
"mul":lambda x,y:x*y,
"div":lambda x,y:x/y
}
calculatorDict.get("add",lambda x , y:None)(2,3)
# because we are repeating x,y in all lambda so better approach
def calculatorCorrected(operator,x,y):
return {
"add":lambda :x+y,
"sub":lambda :x-y,
"mul":lambda :x*y,
"div":lambda :x/y
}.get(operator , lambda :"None")()
calculatorCorrected("add",2,3)
# How to merge multiple dictionaries
x = {1:2,3:4}
y = {3:5,6:7}
{**x,**y}
# how to merge multiple list you gussed it correct
a = [1,2,3]
b=[2,3,4,5]
[*a,*b]
# Named tuple
from collections import namedtuple
from sys import getsizeof
vector = namedtuple("Vector" , ["x","y","z","k"])(11,12,212,343)
vector,vector[0], vector.y # can be accessed lke list and dic
# how to manage a dictionary with count
from collections import Counter
pubg_level3_bag = Counter()
kill = {"kar98":1 , "7.76mm":60}
pubg_level3_bag.update(kill)
print(pubg_level3_bag)
more_kill = {"7.76mm":30 , "scarl":1 , "5.56mm":30}
pubg_level3_bag.update(more_kill)
print(pubg_level3_bag)
# don't remove element from the front of a list in python use instead deque
from collections import deque
# for Datastructure with locking functionality use queue module in python
# how to check if the data structure is iterable
from collections import Iterable
isinstance([1,2,3] , Iterable)