List comprehensions (advanced)
List comprehensions are a powerful and expressive way to create lists in Python using a single line of code. While basic list comprehensions handle simple transformations, advanced list comprehensions allow for more complex logic, including conditional filtering, nested loops, and function application.
Basic Structure (Recap)
[expression for item in iterable if condition]
1. Conditional Logic (if…else inside)
You can use an inline if…else in the expression part to choose different values:
# Label even/odd numbers
result = [‘even’ if x % 2 == 0 else ‘odd’ for x in range(5)]
# Output: [‘even’, ‘odd’, ‘even’, ‘odd’, ‘even’]
Note: if…else must go before the for loop, unlike a simple if filter which comes after.
2. Nested Loops
List comprehensions can handle multiple loops just like nested for loops.
# Cartesian product
pairs = [(x, y) for x in [1, 2] for y in [‘a’, ‘b’]]
# Output: [(1, ‘a’), (1, ‘b’), (2, ‘a’), (2, ‘b’)]
Loop order follows the same nesting as regular for-loops.
3. Using Functions in List Comprehensions
Apply functions directly inside the comprehension:
def square(x):
return x * x
squares = [square(i) for i in range(5)]
# Output: [0, 1, 4, 9, 16]
You can also use built-in functions or lambdas:
words = [“hello”, “world”, “python”]
lengths = [len(word) for word in words]
# Output: [5, 5, 6]
4. Combining Multiple Conditions
Add multiple if statements for filtering:
# Numbers divisible by both 2 and 3
nums = [x for x in range(20) if x % 2 == 0 if x % 3 == 0]
# Output: [0, 6, 12, 18]
5. Flattening Nested Lists
List comprehensions can flatten 2D lists:
matrix = [[1, 2], [3, 4], [5, 6]]
flattened = [num for row in matrix for num in row]
# Output: [1, 2, 3, 4, 5, 6]
6. With Enumerate or Zip
You can also use enumerate() or zip() for more advanced use cases:
Summary:
Advanced list comprehensions allow:
- Conditional value assignment (if…else)
- Nested loops
- Function application
- Filtering with multiple conditions
- Flattening nested lists
- Integration with built-ins like zip(), enumerate()
Generators and iterators
Generators and Iterators – Detailed Explanation
Generators and iterators are powerful tools in Python for working with sequences of data without loading everything into memory. They are essential when dealing with large datasets, streams, or lazy evaluation.
What is an Iterator?
- An iterator is an object that implements the iterator protocol:
- It has a __iter__() method (returns the iterator object itself).
- It has a __next__() method (returns the next value, or raises StopIteration when done).
Example:
nums = iter([1, 2, 3])
print(next(nums)) # 1
print(next(nums)) # 2
print(next(nums)) # 3
# next(nums) # Raises StopIteration
You can create custom iterators by defining classes with __iter__() and __next__() methods.
What is a Generator?
A generator is a simpler way to create an iterator using functions and the yield keyword. Unlike return, yield pauses the function and saves its state for resumption.
Generator Function Example:
def count_up_to(n):
i = 1
while i <= n:
yield i
i += 1
counter = count_up_to(3)
for num in counter:
print(num)
Output:
1
2
3
Each call to next(counter) continues from where it left off after yield.
Generator Expressions
Like list comprehensions but more memory-efficient because they generate items on the fly.
gen = (x * x for x in range(5))
print(next(gen)) # 0
print(next(gen)) # 1
Why Use Generators?
- Memory-efficient: They don’t store the entire sequence in memory.
- Lazy evaluation: They compute values only when needed.
- Composability: Easily chain multiple generators.
When to Use Iterators vs Generators
| Feature | Iterator (Manual) | Generator (Function/yield) |
|---|---|---|
| Syntax | Requires class with __iter__() and __next__() |
Uses function with yield statements |
| Implementation | Explicit iteration protocol | Implicit iteration via generator protocol |
| Use Case | Complex iteration logic with custom state | Lazy evaluation and memory-efficient sequences |
| Memory | Typically stores complete collection | Generates items on-demand (memory efficient) |
| Performance | Slightly faster for small collections | Better for large/infinite sequences |
Custom Iterator Example
class Countdown:
def __init__(self, start):
self.current = start
def __iter__(self):
return self
def __next__(self):
if self.current <= 0:
raise StopIteration
self.current -= 1
return self.current + 1
Summary:
- Iterators follow a protocol (__iter__, __next__) and require more boilerplate.
- Generators simplify iteration with yield, ideal for streaming or large data.
- Generator expressions are concise and lazy alternatives to list comprehensions.
Decorators
Python Decorators – Detailed Explanation
A decorator in Python is a powerful and elegant way to modify or extend the behavior of functions or methods without changing their actual code. Decorators use higher-order functions (functions that return or accept other functions) and are often used for logging, access control, timing, memoization, and more.
Basics of Functions as First-Class Objects
In Python, functions can be:
- Passed as arguments to other functions
- Returned from other functions
- Assigned to variables
- This is the foundation of decorators.
What is a Decorator?
A decorator is a function that takes another function as input, adds some functionality, and returns it.
Basic Example:
def my_decorator(func):
def wrapper():
print(“Before the function runs.”)
func()
print(“After the function runs.”)
return wrapper
@my_decorator
def greet():
print(“Hello!”)
greet()
Output:
Before the function runs.
Hello!
After the function runs.
The @my_decorator syntax is shorthand for:
greet = my_decorator(greet)
Writing a Decorator That Accepts Arguments
def repeat(num_times):
def decorator(func):
def wrapper(*args, **kwargs):
for _ in range(num_times):
func(*args, **kwargs)
return wrapper
return decorator
@repeat(3)
def say_hello():
print(“Hello!”)
say_hello()
Output:
Hello!
Hello!
Hello!
Common Use Cases
1. Logging:
def logger(func):
def wrapper(*args, **kwargs):
print(f”Calling {func.__name__} with {args}, {kwargs}”)
return func(*args, **kwargs)
return wrapper
2. Timing:
import time
def timer(func):
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
end = time.time()
print(f”{func.__name__} took {end – start:.4f} seconds”)
return result
return wrapper
3. Authentication / Authorization (often in web apps).
Using functools.wraps
Always use @functools.wraps(func) in your wrapper to preserve the original function’s name, docstring, and metadata:
from functools import wraps
def my_decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
print(“Wrapped function”)
return func(*args, **kwargs)
return wrapper
Summary:
- A decorator is a callable that takes a function and returns a modified version of it.
- Use @decorator_name syntax to apply a decorator.
- They support use cases like logging, access control, caching, validation, and performance monitoring.
- Decorators can be stacked, parameterized, and used on methods or classes too.
Context managers
Context Managers in Python – Detailed Explanation
A context manager in Python is a construct that properly manages resources (like files, network connections, or database sessions), ensuring they are acquired and released cleanly — even if errors occur.
They are commonly used with the with statement for automatic setup and teardown of resources.
Why Use a Context Manager?
- Ensures resources are freed correctly, even in case of exceptions.
- Helps avoid issues like file leaks, open sockets, or unclosed connections.
- Makes code cleaner and more readable.
Common Use Case: File Handling
with open(‘data.txt’, ‘r’) as f:
content = f.read()
# File is automatically closed here
This is cleaner and safer than:
f = open(‘data.txt’, ‘r’)
try:
content = f.read()
finally:
f.close()
How Does It Work?
- Behind the scenes, a context manager is any object that implements:
- __enter__() – sets things up and returns the resource.
- __exit__(exc_type, exc_value, traceback) – cleans up the resource.
Creating a Custom Context Manager (Using a Class)
class OpenFile:
def __init__(self, filename, mode):
self.filename = filename
self.mode = mode
self.file = None
def __enter__(self):
self.file = open(self.filename, self.mode)
return self.file
def __exit__(self, exc_type, exc_val, exc_tb):
if self.file:
self.file.close()
Usage:
with OpenFile(‘sample.txt’, ‘w’) as f:
f.write(‘Hello, world!’)
Cleaner Approach: contextlib Module
Python provides a simpler way to write context managers using contextlib.contextmanager.
from contextlib import contextmanager
@contextmanager
def open_file(name, mode):
f = open(name, mode)
try:
yield f
finally:
f.close()
Real-World Use Cases
- File I/O (open)
- Locking mechanisms (e.g., threading)
- Database connections (like with connection: in SQLAlchemy)
- Managing resources in testing (setup/teardown)
- Suppressing exceptions or logs (with contextlib.suppress())
Summary:
- Context managers manage setup and teardown automatically.
- They’re used with the with statement.
- Create one using a class (__enter__ / __exit__) or with @contextmanager.
- Ensure safe handling of files, resources, or operations where cleanup is crucial.