Assembly Line Syntax

It's often convenient to imagine a process as a tree-shaped (or graph-shaped) assembly line. Inputs flow in through the leaves and are transformed, filtered or reduced where branches join. This well represents lazy-functional programming. Say one wants the words from a line-delimited file, excluding acronyms, then converted to lower case and stored in a set for quick lookup. Compare syntaxes...

English:

1. Get every line in file words.
2. Skip empty lines.
3. Strip white space.
4. Filter out words that have every letter capitalized.
5. Covert each word to lowercase.
6. Add to the set.

Procedural:

words = set()
for line in file('words'):
 line = line.strip()
 if line and not line.isupper():
 words.add(line.lower())

Variables describe edges in a graph, connecting parts of code, but without being obvious to the eye. Names are boring to create and are often misspelled.

Lazy-functional using standard Python:

set(imap(str.lower, ifilterfalse(lambda w: w.isupper(), ifilter(None, imap(str.strip, file('words'))))))

Lazy using a Python iterator comprehension:

set(line.strip().lower() for line in file('words') if line.strip() and not line.isupper())

This wastes time on a second strip() call.

Lazy-functional mixed with a Python iterator comprehension:

set(line.lower() for line in ifilter(None, imap(str.strip, file('words'))) if not line.isupper())

This eliminates the second strip() call but has a confusing mix of syntaxes. The data flow repeatedly changes direction.

Prefix syntax - that of Lisp, Python, C, etc. - is backwards: the functions called later must precede those called earlier. One has to say what comes last first, like saying: "Eat a sandwich; make a sandwich; buy groceries." The words are spelled left to right but the sentence is right to left.

Consider this Python syntax hack to write lazy-functional code like an assembly line.

Lazy-functional using pipes.py:

file('words') | pmethod('strip') | pfilter | pnfilter(methodcaller('isupper')) | pmethod('lower') | pset

This isn't shorter, but allows sequences of operations to be written naturally from left to right, without nested and distant parenthesis.

pmethod() eases mapping using a method of the objects to be mapped, making this version more general because it works if the strings are a mix of str and unicode instances. methodcaller() similarly eases filtering.

Presumably, this would be better in Lisp, using its read macros, but this abuse of __ror__ hasn't caused problems yet.

Related: ASPN Cookbook Python recipe, Dataflow programming.