Is It Possible To Call A Function From Within A List Comprehension Without The Overhead Of Calling The Function?

In this trivial example, I want to factor out the i < 5 condition of a list comprehension into it's own function. I also want to eat my cake and have it too, and avoid the overh

Solution 1:

Consolidating all of the excellent answers in the comments into one.

As georg says, this sounds like you are looking for a way to inline a function or an expression, and there is no such thing in CPython attempts have been made: https://bugs.python.org/issue10399

Therefore, along the lines of "metaprogramming", you can build the lambda's inline and eval:

from typing import Callable
import dis

def b():
    # list comprehension without function call
    return [i for i in range(10) if i < 5]

def gen_list_comprehension(expr: str) -> Callable:
    return eval(f"lambda: [i for i in range(10) if {expr}]")

a = gen_list_comprehension("i < 5")
dis.dis(a.__code__.co_consts[1])
print("=" * 10)
dis.dis(b.__code__.co_consts[1])

which when run under 3.7.6 gives:

60 BUILD_LIST               02 LOAD_FAST                0 (.0)
        >>    4 FOR_ITER                16 (to22)
              6 STORE_FAST               1 (i)
              8 LOAD_FAST                1 (i)
             10 LOAD_CONST               0 (5)
             12 COMPARE_OP               0 (<)
             14 POP_JUMP_IF_FALSE        416 LOAD_FAST                1 (i)
             18 LIST_APPEND              220 JUMP_ABSOLUTE            4
        >>   22 RETURN_VALUE
==========
  10 BUILD_LIST               02 LOAD_FAST                0 (.0)
        >>    4 FOR_ITER                16 (to22)
              6 STORE_FAST               1 (i)
              8 LOAD_FAST                1 (i)
             10 LOAD_CONST               0 (5)
             12 COMPARE_OP               0 (<)
             14 POP_JUMP_IF_FALSE        416 LOAD_FAST                1 (i)
             18 LIST_APPEND              220 JUMP_ABSOLUTE            4
        >>   22 RETURN_VALUE

From a security standpoint "eval" is dangerous, athough here it is less so because what you can do inside a lambda. And what can be done in an IfExp expression is even more limited, but still dangerous like call a function that does evil things.

However, if you want the same effect that is more secure, instead of working with strings you can modify AST's. I find that a lot more cumbersome though.

A hybrid approach would be the call ast.parse() and check the result. For example:

import ast
def is_cond_str(s: str) -> bool:
    try:
        mod_ast = ast.parse(s)
        expr_ast = isinstance(mod_ast.body[0])
        if not isinstance(expr_ast, ast.Expr):
            returnFalsecompare_ast= expr_ast.value
        if not isinstance(compare_ast, ast.Compare):
            return False
        return True
    except:
        return False

This is a little more secure, but there still may be evil functions in the condition so you could keep going. Again, I find this a little tedious.

Coming from the other direction of starting off with bytecode, there is my cross-version assembler; see https://pypi.org/project/xasm/