¿What is funcional programming?

In computer science, functional programming is a programming paradigma style of building the structure and elements of computer programs—that treats computation as the evaluation of mathematical functions and avoids changing-state and mutable data. It is a declarative programming paradigm, which means programming is done with expressions or declarations instead of statements. (...) can make it much easier to understand and predict the behavior of a program, which is one of the key motivations for the development of functional programming.

Functional programming
from Wikipedia

Imperative vs Declarative (Functional)

(HOW vs WHAT?)

Calculate partially invalid string with operation.

              input = "23+45++++2++5++32++100"
            

Actions that change state from initial state to resultI => Imperative style => HOW?

              input = "23+45++++2++5++32++100"

              res = 0
              for t in input.split("+"):
                  if t:
                    res += int(t)

              print(res)
            

              ["23", "45", "", "", "", "2", "", "5", "", "32", "", "100"], 0
              "23", 0
              "45", 23
              "2", 68
              "5", 70
              "32", 75
              "100", 107
              207
            

Apply rules, restrictions, transformation (and compositions) => Declarative style => WHAT

              input = "23+45++++2++5++32++100"

              from functools import reduce
              from operator import add

              res = reduce(add, map(int, filter(bool, input.split("+"))))

              print(res)
            

              ["23", "45", "", "", "", "2", "", "5", "", "32", "", "100"]
              ["23", "45", "2", "5", "32", "100"]
              [23, 45, 2, 5, 32, 100]
              207
            

What FP means?

• No mutable data, and no (uncontrolled) side effect
• No (implicit/hidden) state, FP do not eliminate state, it just make it visible and explicit (at least when programmers want it to be)
• Function as first-class citizen, like strings or numbers, and you can operate with them (higher order func, composition...)
• Functions are pure functions in the mathematical sense: same output for the same inputs
• Lazy evaluation
• Recursion (with TCO)
• Iterators, sequences, pattern matching, monads....

Why do we want to be functional?

• Cleaner code: we don't have to follow the change of state to comprehend what a function, a, method, a class, a whole project works.
• Referential transparency: Expressions can be replaced by its values. If we call a function with the same parameters, we know for sure the output will be the same.
  • Memoization: cache
  • Parallelization: functions calls are independent, no sync needed => concurrence simpler)
  • Better modularization: easy to plug/unpplug
  • Easy of test
  • Ease of debugging

Python is a multi paradigm programming language so

What does Python offer
to make our code more functional?

List comprenhension

Calculate the combinations of the rock-paper-scissors game for which there is a winner.

              values = ['rock', 'paper', 'scissors']
              combs = []
              for x in values:
                  for y in values:
                      if x != y:
                          combs.append((x, y))
              print(combs)
            

              [('rock', 'paper'), ('rock', 'scissors'), ('paper', 'rock'),
               ('paper', 'scissors'), ('scissors', 'rock'), ('scissors', 'paper')]
            

              values = ['rock', 'paper', 'scissors']
              combs = [(x, y) for x in values for y in values if x != y]
              print(combs)
            

              [('rock', 'paper'), ('rock', 'scissors'), ('paper', 'rock'),
               ('paper', 'scissors'), ('scissors', 'rock'), ('scissors', 'paper')]
            

Are you sure you can not do it better?

Iterators & Generators

iterable: An object capable of returning its members one at a time. Examples of iterables include all sequence types (such as list, str, and tuple) and some non-sequence types like dict, file objects, and objects of any classes you define with an __iter__() method or __getitem__ (...) When an iterable object is passed as an argument to the built-in function iter(), it returns an iterator for the object. This iterator is good for one pass over the set of values.

              for n in [1, 2, 3]:
                  print (n)
            

              1
              2
              3
            

              for l in "abc":
                  print (l)
            

              a
              b
              c
            

              file = open("to_pdf.sh", "r")
              for line in file:
                  print(line)
            

              #!/bin/bash
              node node_modules/decktape/decktape.js --no-sandbox reveal "http://localhost:8000" slides.pdf
            

__iter__:This method is called when an iterator is required for a container. This method should return a new iterator object that can iterate over all the objects in the container. For mappings, it should iterate over the keys of the container.

iterator: An object representing a stream of data. Repeated calls to the iterator’s __next__() method (or passing it to the built-in function next()) return successive items in the stream. When no more data are available a StopIteration exception is raised instead. At this point, the iterator object is exhausted and any further calls to its __next__() method just raise StopIteration again. Iterators are required to have an __iter__() method that returns the iterator object itself so every iterator is also iterable and may be used in most places where other iterables are accepted.

              class yrange:
                  def __init__(self, n):
                      self.i = 0
                      self.n = n

                  def __iter__(self):
                      return self

                  def __next__(self):
                      if self.i < self.n:
                          i = self.i
                          self.i += 1
                          return i
                      else:
                          raise StopIteration()
            

              y = yrange(3)
              print(next(y))
              print(next(y))
              print(next(y))
              print(next(y))
            

              0
              1
              2
              ---------------------------------------------------------------------------
              StopIteration                             Traceback (most recent call last)
              <ipython-input-49-c97cadfde7e4> in <module>()
                    3 print(next(y))
                    4 print(next(y))
              ----> 5 print(next(y))

              <ipython-input-45-79e514da8e3b> in __next__(self)
                   13             return i
                   14         else:
              ---> 15             raise StopIteration()

              StopIteration:
            

              for n in yrange(3):
                print(n)
              0
              1
              2
            

              def my_for(iterable):
                  it = iter(iterable)
                  while True:
                      try:
                          print(next(it))
                      except StopIteration:
                          break

              my_for(yrange(3))
              0
              1
              2
            

Iterator is good for one pass over the set of values.

              y = yrange(5)
              print(tuple(y))
              print(tuple(y))
            

              (0, 1, 2, 3, 4)
              ()
            

generator A function which returns a generator iterator. It looks like a normal function except that it contains yield expressions for producing a series of values usable in a for-loop or that can be retrieved one at a time with the next() function.

generator iterator An object created by a generator function.

              def my_generator():
                yield 1
                yield 2
                yield 3
            

              g = my_generator()
              print(g)
              print(next(g))
              print(next(g))
              print(next(g))
              print(next(g))
            

              <generator object my_generator at 0x7ff9ec48fc50>
              1
              2
              3
              ---------------------------------------------------------------------------
              StopIteration                             Traceback (most recent call last)
              <ipython-input-55-567ce73dea18> in <module>()
                    4 print(next(g))
                    5 print(next(g))
              ----> 6 print(next(g))

              StopIteration:
            

              def count(stop):
                  """Return all numbers <= stop."""

                  numbers = []
                  n = 0
                  while n <= stop:
                      numbers.append(n)
                      n += 1
                  return numbers
            

              print(count(10))

              [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
            

              print(count(1e18))

              ===================================================================
              | Kernel restarting                                               |
              |                                                                 |
              | The kernel appears to have died. It will restart automatically. |
              ===================================================================
            

              def count():
                  n = 0
                  while True:
                      yield n
                      n +=1
            

              counter = count()
              print(next(counter))
              print(next(counter))
              print(next(counter))
              print(next(counter))
            

              0
              1
              2
              3
            

              for num in count():
                  print(num)

              # zZzZzZzZz...
            

Let's go back to the previous example...

              values = ['rock', 'paper', 'tijer ']
              combs = ((x, y) for x in values for y in values if x != y)
              print(combs)
            

              <generator object <genexpr> at 0x7ff9ec545468>
            

              next(combs)
              ('rock', 'paper')

              next(combs)
              ('rock', 'scissors')
            

In summary:

• An iterable is an object capable of returning its members one by one. Implement the __ite __() or __getitem__() method.
• An iterator is an iterable object since it is returned to itself that also adds the method __next__().
• Iterators are lazy (calculated when they are needed).
• Iterators are for a single use.
• A generator function is a function capable of returning values one by one (using yield), returning a generator.
• A generator is a kind of iterator generated by a generator function.
• A generator expression is like a list comprenhension but with the benefits of the generators.

If you need mutch more, read "Generator Tricks for Systems Programmers" by David M. Beazley
and some Python Enhancement Proposals PEP 288 , PEP 325 , PEP 342 and PEP 380.

itertools

This module implements a number of iterator building blocks inspired by constructs from APL, Haskell, and SML. Each has been recast in a form suitable for Python.

The module standardizes a core set of fast, memory efficient tools that are useful by themselves or in combination. Together, they form an “iterator algebra” making it possible to construct specialized tools succinctly and efficiently in pure Python.

Infinite iterators:

count(), cycle(), repeat()

Iterators terminating on the shortest input sequence:

accumulate(), chain(), chain.from_iterable(), compress(), dropwhile(), filterfalse(), groupby(), islice(), starmap(), takewhile(), tee(), zip_longest()

Combinatoric iterators:

product(), permutations(), combinations(), combinations_with_replacement()

See itertools doc.

Ex.: A (rudimentary) compression algorithm

Let's implement a very basic string compression function using the counts of repeated characters.
For example:

              compress("sssdddddxxaaaaaa")
              "s3d5x2a6"
            

...without itertools

              def compress(word):
                  result = []
                  current = word[0]
                  counter = 1

                  for letter in word[1:]:
                      if letter == current:
                          # We're still in the same group
                          counter += 1
                      else:
                          # We need to start a new group
                          result += [current, str(counter)]
                          current = letter # start a new group
                          counter = 1
                  result += [current, str(counter)]
                  return "".join(result)


              compress("sssdddddxxaaaaaa")
              "s3d5x2a6"
            

...with itertools

itertools.groupby(iterable, key=None)

Make an iterator that returns consecutive keys and groups from the iterable. The key is a function computing a key value for each element. If not specified or is None, key defaults to an identity function and returns the element unchanged. Generally, the iterable needs to already be sorted on the same key function.

The operation of groupby() is similar to the uniq filter in Unix. It generates a break or new group every time the value of the key function changes (which is why it is usually necessary to have sorted the data using the same key function). That behavior differs from SQL’s GROUP BY which aggregates common elements regardless of their input order.

              def compress(word):
                  return ''.join(f"{key}{len(tuple(group))}"
                                 for key, group in itertools.groupby(word))


              compress("sssdddddxxaaaaaa")
              "s3d5x2a6"
           

Ex.: Rolling dices

If we roll four six-sided dices, what are their posible outcomes?

                def product(first, second, third, fourth):
                    """A generator of the Cartesian product of four iterables."""
                    for w in first:
                        for x in second:
                            for y in third:
                                for z in fourth:
                                    yield (w, x, y, z)


                dice = range(1, 7)
                len(tuple(product(dice, dice, dice, dice)))
                1296
              

...with itertools

itertools.product(*iterables, repeat=1)

Cartesian product of input iterables.

(...) To compute the product of an iterable with itself, specify the number of repetitions with the optional repeat keyword argument. For example, product(A, repeat=4) means the same as product(A, A, A, A).

              import itertools
              dice = range(1, 7)
              len(tuple(itertools.product(dice, repeat=4)))
              1296
            

...and in how many outcomes they add up to 6?

              tuple(filter(lambda x: sum(x) == 6,
                           itertools.product(dice, repeat=4)))
            

              ((1, 1, 1, 3), (1, 1, 2, 2), (1, 1, 3, 1), (1, 2, 1, 2), (1, 2, 2, 1),
               (1, 3, 1, 1), (2, 1, 1, 2), (2, 1, 2, 1), (2, 2, 1, 1), (3, 1, 1, 1))
            

To know more about irtertools see

Kung Fu at Dawn with Itertools
by Víctor Terrón (@pyctor)
at EuroPython 2016

Video EN / Video ES / Repo [CC BY-SA 2.0]

Lambda functions

lambda: An anonymous inline function consisting of a single expression which is evaluated when the function is called. The syntax to create a lambda function is lambda [arguments]: expression

expression: A piece of syntax which can be evaluated to some value. In other words, an expression is an accumulation of expression elements like literals, names, attribute access, operators or function calls which all return a value.

                sum_numbers = lambda x, y: x+y
              

                print(sum_numbers)
                <function <lambda> at 0x7f0aef3d9ea0>
              

                sum_numbers(1, 4)
                5
              

              even_or_odd = lambda x: "even" if x % 2 == 0 else "odd"

              print(f"1 is {even_or_odd(1)}")
              print(f"4 is {even_or_odd(4)}")
              print(f"22 is {even_or_odd(22)}")
              print(f"1479 is {even_or_odd(1479)}")
            

              1 is odd
              4 is even
              22 is even
              1479 is odd
            

Higher-order functions

In mathematics and computer science, a higher-order function (also functional, functional form or functor) is a function that takes one or more functions as arguments or return a function.

Higher-order function
from Wikipedia

              def twice(f):
                  return lambda x: f(f(x))

              plus_three = lambda x: x + 3

              twice_plus_three = twice(plus_three)
            

              twice_plus_three(10)
              16
            

Python comes with some higher-order functions

map(function, iterable, ...)

Return an iterator that applies function to every item of iterable, yielding the results. If additional iterable arguments are passed, function must take that many arguments and is applied to the items from all iterables in parallel. With multiple iterables, the iterator stops when the shortest iterable is exhausted. For cases where the function inputs are already arranged into argument tuples, see itertools.starmap().

              tuple(map(lambda x, y: (x,y), (1, 2, 3, 4), ('a', 'b', 'c', 'd')))

              ((1, 'a'), (2, 'b'), (3, 'c'), (4, 'd'))
            

filter(function, iterable)

Construct an iterator from those elements of iterable for which function returns true. iterable may be either a sequence, a container which supports iteration, or an iterator. If function is None, the identity function is assumed, that is, all elements of iterable that are false are removed.

Note that filter(function, iterable) is equivalent to the generator expression (item for item in iterable if function(item)) if function is not None and (item for item in iterable if item) if function is None.

See itertools.filterfalse() for the complementary function that returns elements of iterable for which function returns false.

              tuple(filter(lambda x: x % 2 == 0, [1, 2, 3, 4, 5, 15, 17, 32, 1986]))

              (2, 4, 32, 1986)
            

sorted(iterable, *, key=None, reverse=False)

Return a new sorted list from the items in iterable.

Has two optional arguments which must be specified as keyword arguments.

key specifies a function of one argument that is used to extract a comparison key from each list element: key=str.lower. The default value is None (compare the elements directly).

reverse is a boolean value. If set to True, then the list elements are sorted as if each comparison were reversed.

The built-in sorted() function is guaranteed to be stable. A sort is stable if it guarantees not to change the relative order of elements that compare equal — this is helpful for sorting in multiple passes (for example, sort by department, then by salary grade).

              from collections import namedtuple

              Student = namedtuple('Student', ('name', 'age', 'grade'))

              students = (
                  Student("María", 21, 'C'),
                  Student("Pedro", 22, 'B'),
                  Student("Rosa", age=21, grade='A')
              )

              sorted(students, key=lambda s: s.age)
            

              [Student(name='María', age=21, grade='C'),
               Student(name='Rosa', age=21, grade='A'),
               Student(name='Pedro', age=22, grade='B')]
            

              from operator import attrgetter
              sorted(students, key=attrgetter('age', 'grade'))
            

              [Student(name='Rosa', age=21, grade='A'),
               Student(name='María', age=21, grade='C'),
               Student(name='Pedro', age=22, grade='B')]
            

functools

The functools module is for higher-order functions: functions that act on or return other functions. In general, any callable object can be treated as a function for the purposes of this module.

See functools doc.

functools.reduce(function, iterable[, initializer])

Apply function of two arguments cumulatively to the items of sequence, from left to right, so as to reduce the sequence to a single value. For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5). The left argument, x, is the accumulated value and the right argument, y, is the update value from the sequence. If the optional initializer is present, it is placed before the items of the sequence in the calculation, and serves as a default when the sequence is empty. If initializer is not given and sequence contains only one item, the first item is returned.

              import functools
              functools.reduce(lambda x, y: x * y, range(1, 11))
            

@functools.lru_cache(maxsize=128, typed=False)

Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments. (...) If maxsize is set to None, the LRU feature is disabled and the cache can grow without bound. The LRU feature performs best when maxsize is a power-of-two. If typed is set to true, function arguments of different types will be cached separately.

              import functools

              @functools.lru_cache()
              def get_heavy_func(param):
                  print(f"Multiply (param={param} by 3")
                  return param * 3

            

              print(get_heavy_func(1))
              Multiply 1 by 3
              3

              print(get_heavy_func(2))
              Multiply 2 by 3
              6

              print(get_heavy_func(1))
              3

              print(get_heavy_func(2))
              6
            

Clousures and decorators

              class MultiplierMaker:
                  def __init__(self, n):
                      self.n = n

                  def multiplier(self, x):
                      return self.n * x
            

              times3 = MultiplierMaker(3)
              times5 = MultiplierMaker(5)

              print(times3.multiplier(9))
              27
              print(times5.multiplier(3))
              15
              print(times5.multiplier(times3.multiplier(2)))
              30
            

A closure is a function evaluated in an environment that contains
one or more variables dependent on another environment.

              def make_multiplier_of(n):
                  def multiplier(x):
                      return x * n
                  return multiplier
            

              times3 = make_multiplier_of(3)
              times5 = make_multiplier_of(5)

              print(times3(9))
              27
              print(times5(3))
              15
              print(times5(times3(2)))
              30
            

              print(make_multiplier_of.__closure__)
              None
              print(times3.__closure__[0].cell_contents)
              3
            

Decorators are “wrappers”, which means that they let you execute code before
and after the function they decorate without modifying the function itself.

A decorator is a function that takes in another function and returns another function.

The best exaplanation about decorators in Python is here

              def bold(fn):
                  def wrapped():
                      return f"**{fn()}**"
                  return wrapped

              def italic(fn):
                  def wrapped():
                      return f"_{fn()}_"
                  return wrapped
            

              def hello():
                  return "hello world"
            

              print(bold(italic(hello))())
              **_hello world_**
            

              @bold
              @italic
              def hello():
                  return "hello world"
            

              print(hello())
              **_hello world_**
            

There is a small problem with decorated functions and their metadata

              @bold
              def hello():
                  """Print hello message"""
                  return "hello world"
            

              print(hello.__name__)
              wrapped
              print(hello.__doc__)
              None
            

            import functools

            def bold(fn):
                @functools.wraps(fn)
                def wrapped():
                    return f"*{fn()}*"
                return wrapped

            @bold
            def hello():
                """Print hello message"""
                return "hello world"
            

              print(hello.__name__)
              hello
              print(hello.__doc__)
              Print hello message
            

              from functools import wraps

              def md_tag(tag):
                  def factory(func):
                      @wraps(func)
                      def decorator(msg):
                          return f"{tag}{func(msg)}{tag}"
                      return decorator
                  return factory

              @md_tag("**")
              @md_tag("_")
              def message(msg):
                  """Return a text message"""
                  return msg

            

              print(message("Hello T3chFest"))
              **_Hello T3chFest_**

              print(message.__name__)
              message

              print(message.__doc__)
              Return a text message
            

Partial function application

              papply: (((a × b) → c) × a) → (b → c) = λ(f, x). λy. f (x, y)
            

The process of fixing a number of arguments to a function, producing another function of smaller arity.

Partial application
from Wikipedia

              def log(level, msg):
                  print(f"[{level}]: {msg}")

              def debug(msg):
                  log("debug", msg)

              log("debug", "Start doing something")
              log("debug", "Continue with something else")
              debug("Finished. Procastinate")
            

              [debug]: Start doing something
              [debug]: Continue with something else
              [debug]: Finished. Procastinate
            

              from functools import partial

              def log(level, msg):
                  print(f"[{level}]: {msg}")

              debug = partial(log, "debug")

              debug("Start doing something")
              debug("Continue with something else")
              debug("Procastinate")
            

              [debug]: Start doing something
              [debug]: Continue with something else
              [debug]: Procastinate
              [info]: End
            

              info = partial(log, "info")
              warn = partial(log, "warning")
              error = partial(log, "error")
            

              from django.http  import HttpResponse

              JsonResponse = lambda content, *args, **kwargs: HttpResponse(
                  json.dumps(content),
                  content_type="application/json",
                  *args,
                  **kwargs
              )

              JsonOKResponse  = partial(JsonResponse, status=200)
              JsonBadRequestResponse = partial(JsonResponse, status=400)
              JsonCreatedResponse = partial(JsonResponse, status=201)
              JsonNotAllowedResponse = partial(JsonResponse, status=405)
            

              from django.core.mail import send_mail

              email_admin = partial(send_email, email="admin@example.com")
              email_general_it = partial(send_email, email="it@example.com")
              email_marketing = partial(send_email, email="marketing@example.com")
              email_sales = partial(send_email, email="sales@example.com")

              # (...)
              # In our services

              if thing_is_broken():
                   email_admin(subject="It's brokened!", body='Fix it!')

              # (...)

              if sales_people():
                   email_sales(subject="sales stuff", body="business, business, business!")
            

Currying

              curry: ((a × b) → c) → (a → (b → c)) = λf. λx. λy. f (x, y)
            

The technique of transforming a function that takes multiple arguments in such a way that it can be called as a chain of functions each with a single argument.

Partial application
from Wikipedia

              def curry(fn):
                  def curried(*args, **kwargs):
                      return curry(partial(fn, *args, **kwargs)) if args or kwargs else fn()
                  return curried
            

              @curry
              def accumulator(*args):
                  return sum(args)

              acc = accumulator(10)
              acc = acc(12)(15)(22)
              acc = acc(1)(1)

              print(acc())
            

              61
            

              currencies =  {
                  'GBP': 0.879700,
                  'JPY': 131.259995,
                  'EUR': 1.0,
                  'USD': 1.223200
              }

              @curry
              def exchange(from_currency, to_currency, amount):
                  return amount * currencies[to_currency] / currencies[from_currency]

              from_eur = exchange('EUR')
              from_gbp = exchange('GBP')

              from_eur_to_gbp = from_eur('GBP')
              from_eur_to_usd = from_eur('USD')
              from_gbp_to_eur = from_gbp('EUR')

              print(from_eur_to_gbp(100.0)())
              print(from_gbp_to_eur(87.97)())
              print(from_eur_to_usd(10.0)())
            

              87.97
              100.0
              12.232000000000001
            

Recursion

Does Python have TCO?

I recently posted an entry in my Python History blog on the origins of Python's functional features. A side remark about not supporting tail recursion elimination (TRE) immediately sparked several comments about what a pity it is that Python doesn't do this, including links to recent blog entries by others trying to "prove" that TRE can be added to Python easily. So let me defend my position (which is that I don't want TRE in the language). If you want a short answer, it's simply unpythonic. Here's the long answer: (...)

Tail Recursion elimination
Posted by Guido van Rossum on April 22, 2009

              import sys
              sys.getrecursionlimit()
            

              3000
            

              def fib(n, sum):
                  return sum if not n else fib(n-1, sum+n)

              fib(3500, 0)
            

              <ipython-input-117-6cb630841faa> in fib(n, sum)
                         1 def fib(n, sum):
              ----> 2     return sum if not n else fib(n-1, sum+n)
                         3
                         4 fib(3500, 0)

              RecursionError: maximum recursion depth exceeded
            

              def fib(n):
                  a, b = 0, 1
                  for i in range(n):
                      yield a
                      a, b = b, a + b

              tuple(fib(3500))
            

Python & FP

What about community libs?

• General: cytoolz, funcy, fn.py, hask, Effect, Pydash, Underscore.py, pyramda, PyMonad, pyMonet...
• Immutable / persistent data structures: Pyrsistent, Funktown...
• Specialized: Transducers, Tranducers-Python, RxPy...

More resources at Awesome Functional Python.

Recomendations

• Think in functional, embraces declarative programming.
• Start using everything shown in this talk: iterators/generators, itertools, functools, collections, operator, decorators...
• Global variables are evil, minimize its use
• Make more pure functions, easier to test and more reusable
• Less classes, classes ofuscate function calls. So make MORE FUNCTIONS.
  Stop Writing Classes at PyCon US '12 by Jack Diederich)

...and

                import this
              

Any question?