Intermediate Topics

List comprehensions (advanced)

List comprehensions are a powerful and expressive way to create lists in Python using a single line of code. While basic list comprehensions handle simple transformations, advanced list comprehensions allow for more complex logic, including conditional filtering, nested loops, and function application. This technique remains a staple in modern Python for writing concise, readable code, especially in data processing tasks.

Basic Structure (Recap)

[expression for item in iterable if condition]

1. Conditional Logic (if…else inside)

You can use an inline if…else in the expression part to choose different values:

# Label even/odd numbers
result = [‘even’ if x % 2 == 0 else ‘odd’ for x in range(5)]
# Output: [‘even’, ‘odd’, ‘even’, ‘odd’, ‘even’]

Note: if…else must go before the for loop, unlike a simple if filter which comes after. For a broader look at how conditional logic interacts with Python control flow, see the Control Flow lesson.

2. Nested Loops

List comprehensions can handle multiple loops just like nested for loops.

# Cartesian product
pairs = [(x, y) for x in [1, 2] for y in [‘a’, ‘b’]]
# Output: [(1, ‘a’), (1, ‘b’), (2, ‘a’), (2, ‘b’)]

Loop order follows the same nesting as regular for-loops. For a deeper dive into loops and iteration, refer to the Loops and Iteration lesson.

3. Using Functions in List Comprehensions

Apply functions directly inside the comprehension:

def square(x):
return x * x

squares = [square(i) for i in range(5)]
# Output: [0, 1, 4, 9, 16]

You can also use built-in functions or lambdas:

words = [“hello”, “world”, “python”]
lengths = [len(word) for word in words]
# Output: [5, 5, 6]

For more on working with functions as first-class objects in Python, see the Introduction to Python lesson.

4. Combining Multiple Conditions

Add multiple if statements for filtering:

# Numbers divisible by both 2 and 3
nums = [x for x in range(20) if x % 2 == 0 if x % 3 == 0]
# Output: [0, 6, 12, 18]

Chaining multiple conditions is common when you need precise filtering. If you want to explore more about conditional logic and control flow, check out the Control Flow lesson.

5. Flattening Nested Lists

List comprehensions can flatten 2D lists:

matrix = [[1, 2], [3, 4], [5, 6]]
flattened = [num for row in matrix for num in row]
# Output: [1, 2, 3, 4, 5, 6]

Flattening is a common pattern when preparing data for processing. For related concepts on list and sequence handling, see the Data Structures lesson.

6. With Enumerate or Zip

You can also use enumerate() or zip() for more advanced use cases:

Summary:

Advanced list comprehensions allow:

• Conditional value assignment (if…else)
• Nested loops
• Function application
• Filtering with multiple conditions
• Flattening nested lists
• Integration with built-ins like zip(), enumerate()
• Using enumerate() or zip() to pair indices with items

Generators and iterators

Generators and Iterators – Detailed Explanation
Generators and iterators are powerful tools in Python for working with sequences of data without loading everything into memory. They are essential when dealing with large datasets, streams, or lazy evaluation.

What is an Iterator?

• An iterator is an object that implements the iterator protocol:
• It has a __iter__() method (returns the iterator object itself).
• It has a __next__() method (returns the next value, or raises StopIteration when done).

Example:

nums = iter([1, 2, 3])

print(next(nums)) # 1
print(next(nums)) # 2
print(next(nums)) # 3
# next(nums) # Raises StopIteration

You can create custom iterators by defining classes with __iter__() and __next__() methods.

What is a Generator?

A generator is a simpler way to create an iterator using functions and the yield keyword. Unlike return, yield pauses the function and saves its state for resumption.

Generator Function Example:

def count_up_to(n):
i = 1
while i <= n:
yield i
i += 1

counter = count_up_to(3)

for num in counter:
print(num)

Output:

1
2
3

Each call to next(counter) continues from where it left off after yield.

Generator Expressions

Like list comprehensions but more memory-efficient because they generate items on the fly.

gen = (x * x for x in range(5))
print(next(gen)) # 0
print(next(gen)) # 1

Why Use Generators?

• Memory-efficient: They don’t store the entire sequence in memory.
• Lazy evaluation: They compute values only when needed.
• Composability: Easily chain multiple generators.

When to Use Iterators vs Generators

Feature	Iterator (Manual)	Generator (Function/yield)
Syntax	Requires class with __iter__() and __next__()	Uses function with yield statements
Implementation	Explicit iteration protocol	Implicit iteration via generator protocol
Use Case	Complex iteration logic with custom state	Lazy evaluation and memory-efficient sequences
Memory	Typically stores complete collection	Generates items on-demand (memory efficient)
Performance	Slightly faster for small collections	Better for large/infinite sequences

Custom Iterator Example

class Countdown:
def __init__(self, start):
self.current = start

def __iter__(self):
return self

def __next__(self):
if self.current <= 0:
raise StopIteration
self.current -= 1
return self.current + 1

Summary:

• Iterators follow a protocol (__iter__, __next__) and require more boilerplate.
• Generators simplify iteration with yield, ideal for streaming or large data.
• Generator expressions are concise and lazy alternatives to list comprehensions.

Decorators

Python Decorators – Detailed Explanation

A decorator in Python is a powerful and elegant way to modify or extend the behavior of functions or methods without changing their actual code. Decorators use higher-order functions (functions that return or accept other functions) and are often used for logging, access control, timing, memoization, and more.

Basics of Functions as First-Class Objects

In Python, functions can be:

• Passed as arguments to other functions
• Returned from other functions
• Assigned to variables
• This is the foundation of decorators.

What is a Decorator?

A decorator is a function that takes another function as input, adds some functionality, and returns it.

Basic Example:

def my_decorator(func):
def wrapper():
print(“Before the function runs.”)
func()
print(“After the function runs.”)
return wrapper

@my_decorator
def greet():
print(“Hello!”)

greet()

Output:

Before the function runs.
Hello!
After the function runs.

The @my_decorator syntax is shorthand for:

greet = my_decorator(greet)

Writing a Decorator That Accepts Arguments

def repeat(num_times):
def decorator(func):
def wrapper(*args, **kwargs):
for _ in range(num_times):
func(*args, **kwargs)
return wrapper
return decorator

@repeat(3)
def say_hello():
print(“Hello!”)

say_hello()

Output:

Hello!
Hello!
Hello!

Common Use Cases

1. Logging:

def logger(func):
def wrapper(*args, **kwargs):
print(f”Calling {func.__name__} with {args}, {kwargs}”)
return func(*args, **kwargs)
return wrapper

2. Timing:

import time

def timer(func):
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
end = time.time()
print(f”{func.__name__} took {end – start:.4f} seconds”)
return result
return wrapper

3. Authentication / Authorization (often in web apps).

Using functools.wraps

Always use @functools.wraps(func) in your wrapper to preserve the original function’s name, docstring, and metadata:

from functools import wraps

def my_decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
print(“Wrapped function”)
return func(*args, **kwargs)
return wrapper

Summary:

• A decorator is a callable that takes a function and returns a modified version of it.
• Use @decorator_name syntax to apply a decorator.
• They support use cases like logging, access control, caching, validation, and performance monitoring.
• Decorators can be stacked, parameterized, and used on methods or classes too.

Context managers

Context Managers in Python – Detailed Explanation

A context manager in Python is a construct that properly manages resources (like files, network connections, or database sessions), ensuring they are acquired and released cleanly — even if errors occur.

They are commonly used with the with statement for automatic setup and teardown of resources.

Why Use a Context Manager?

• Ensures resources are freed correctly, even in case of exceptions.
• Helps avoid issues like file leaks, open sockets, or unclosed connections.
• Makes code cleaner and more readable.

Common Use Case: File Handling

with open(‘data.txt’, ‘r’) as f:
content = f.read()
# File is automatically closed here

This is cleaner and safer than:

f = open(‘data.txt’, ‘r’)
try:
content = f.read()
finally:
f.close()

How Does It Work?

• Behind the scenes, a context manager is any object that implements:
• __enter__() – sets things up and returns the resource.
• __exit__(exc_type, exc_value, traceback) – cleans up the resource.

Creating a Custom Context Manager (Using a Class)

class OpenFile:
def __init__(self, filename, mode):
self.filename = filename
self.mode = mode
self.f = None

def __enter__(self):
self.f = open(self.filename, self.mode)
return self.f

def __exit__(self, exc_type, exc_value, traceback):
if self.f:
self.f.close()
return False

Summary:

• Context managers ensure deterministic setup and teardown of resources.
• They can be implemented via classes or with contextlib for simpler cases.
• Using context managers leads to cleaner, safer resource management.