Engineer's Guide to python-dotenv

Module Map

Design Patterns in Use

1. Strategy Pattern — Variable Resolution Order

The resolve_variables function in main.py:289-311 implements two resolution strategies controlled by the override flag: - override=True: os.environ is checked first, then new_values (file-defined vars). This means file values override existing environment variables. - override=False: new_values is checked first, then os.environ. This means existing environment variables take precedence over file values.

# main.py:301-306
if override:
    env.update(os.environ)
    env.update(new_values)
else:
    env.update(new_values)
    env.update(os.environ)

This is applied inconsistently — see Gotchas below.

2. Template Method / Abstract Base Class — Variable Interpolation

variables.py:18-26 defines Atom as an abstract base class with an abstract resolve(env) method. Two concrete subclasses: - Literal (variables.py:29-45) — returns its value unchanged - Variable (variables.py:48-67) — looks up name in env, falls back to default or ""

Both implement __eq__ and __hash__, making them usable in sets and as dict keys.

3. Context Manager — File Rewriting

The rewrite context manager (main.py:138-190) implements a safe file-update pattern: 1. Opens the source file for reading (or uses StringIO("") if file doesn't exist) 2. Creates a temporary file in the same directory (same filesystem → atomic os.replace) 3. On success: preserves original file mode, then atomically replaces the original 4. On failure: deletes the temp file, re-raises the exception

This is used by both set_key and unset_key.

4. Context Manager — Stream Acquisition

DotEnv._get_stream (main.py:60-73) is a context manager that resolves the input source: - If dotenv_path points to a file/FIFO → open and yield - Else if stream is provided → yield it directly - Else → yield an empty StringIO (silent fallback, logs if verbose)

5. Generator Pattern — Lazy Parsing

DotEnv.parse() (main.py:91-95) yields (key, value) tuples lazily from the stream. The with_warn_for_invalid_lines wrapper (main.py:32-39) wraps the parser's Binding iterator, logging warnings for lines with parse errors while still yielding them (filtered out in parse() by the mapping.key is not None check).

6. Caching — Lazy Dict

DotEnv.dict() (main.py:75-89) caches the parsed result in self._dict. The cache is never invalidated — once computed, it's returned on all subsequent calls. This means calling dict() after modifying the underlying file will return stale data.

Critical Implementation Details

1. dotenv_values Always Uses override=True

This is a non-obvious design decision at main.py:460-467:

return DotEnv(
    dotenv_path=dotenv_path,
    stream=stream,
    verbose=verbose,
    interpolate=interpolate,
    override=True,  # <-- hardcoded True
    encoding=encoding,
).dict()

The dotenv_values function (which returns a dict without modifying os.environ) always passes override=True to DotEnv. This means when resolving ${VAR} references, file-defined variables take precedence over environment variables, even though no environment variables are being modified. This is the correct behavior for a "what would be loaded" preview, but it's a subtle difference from load_dotenv which accepts override as a parameter.

2. find_dotenv Frame Inspection — Fragile by Design

find_dotenv (main.py:332-380) uses sys._getframe() to walk the call stack and find the caller's file location. This is how it discovers which directory to start searching from. The logic at main.py:361-370:

frame = sys._getframe()
current_file = __file__
while frame.f_code.co_filename == current_file or not os.path.exists(
    frame.f_code.co_filename
):
    assert frame.f_back is not None
    frame = frame.f_back
frame_filename = frame.f_code.co_filename
path = os.path.dirname(os.path.abspath(frame_filename))

This skips frames whose filename matches dotenv/main.py itself, then skips frames whose file doesn't exist on disk. This is fragile — if the call stack has unusual frames (e.g., frozen apps, eval'd code), the assertion frame.f_back is not None will crash.

3. _is_interactive Detection — Multiple Heuristics

find_dotenv uses three heuristics to detect interactive mode (main.py:343-354): 1. hasattr(sys, "ps1") or hasattr(sys, "ps2") — standard REPL prompt detection 2. __import__("__main__") and checking if it has __file__ — IPython notebooks don't have __file__ 3. sys.gettrace() is not None — debugger detection

If any of these are true, find_dotenv uses os.getcwd() instead of the caller's file location. This means in a REPL, it searches from the current working directory upward.

4. FIFO Support — Explicit but Limited

The _is_file_or_fifo function (main.py:470-482) explicitly supports FIFO (named pipe) files. This is tested in tests/test_fifo_dotenv.py. However, FIFO support only works for reading — set_key and unset_key use rewrite which calls os.lstat and os.replace, which won't work on a FIFO.

5. set_key Quoting — Custom Logic, Not shlex

The quoting logic in set_key (main.py:214-221) is hand-rolled:

quote = quote_mode == "always" or (
    quote_mode == "auto" and not value_to_set.isalnum()
)
if quote:
    value_out = "'{}'".format(value_to_set.replace("'", "\\'"))

It uses single quotes and escapes single quotes with \'. This is not the same as shlex.quote() (which uses single quotes and handles them by terminating and re-opening). The CLI's list command uses shlex.quote() for shell/export formats (cli.py:108), creating an inconsistency between how values are written and how they're displayed.

6. load_dotenv Returns False When Disabled

When PYTHON_DOTENV_DISABLED is set, load_dotenv returns False (main.py:413-417), which is the same return value as when the file is empty or has no variables. There's no way to distinguish "disabled" from "empty file" from "no variables set."

Entry Points — Start Here

1. src/dotenv/main.py — The Heart

Read this first. It contains: - DotEnv class (lines 42-121) — the core abstraction - load_dotenv (lines 383-430) — the primary entry point for most users - dotenv_values (lines 433-467) — the "preview" function - find_dotenv (lines 332-380) — the file discovery algorithm - set_key/unset_key/get_key (lines 193-286) — the mutation functions - rewrite (lines 138-190) — the safe file-update mechanism - resolve_variables (lines 289-311) — the variable interpolation engine

2. src/dotenv/parser.py — The Tokenizer

Read this second. It implements the actual .env file format parsing. Key sections: - Reader class (lines 69-102) — character-by-character reading with position tracking - parse_key (lines 112-120) — key parsing (single-quoted or unquoted) - parse_value (lines 128-139) — value parsing (single-quoted, double-quoted, unquoted) - parse_binding (lines 142-176) — full KEY=VALUE line parsing - decode_escapes (lines 105-109) — escape sequence handling via codecs.decode

3. src/dotenv/cli.py — The CLI

Read this third. It shows how the library is used in practice: - cli group (lines 38-64) — global options (-f, -q, -e) - list_values (lines 83-109) — output formatting (simple, json, shell, export) - run command (lines 170-201) — subprocess execution with modified environment - Platform-specific run_command — os.execvpe on Unix, subprocess.Popen on Windows

4. src/dotenv/variables.py — The Interpolation Engine

Small but important. The Atom hierarchy and parse_variables generator explain how ${VAR} and ${VAR:-default} are resolved.

5. src/dotenv/__init__.py — The Public API

Short file that defines the public surface. The get_cli_string helper (lines 12-39) is a notable utility for generating shell command strings from Python (e.g., for Fabric tasks).

Known Gotchas

1. override Flag Semantics Are Inverted Between load_dotenv and DotEnv

load_dotenv defaults to override=False (main.py:387), meaning existing environment variables are not overridden. But DotEnv.__init__ defaults to override=True (main.py:50). This means: - load_dotenv() → override=False (safe, doesn't clobber) - DotEnv(dotenv_path=...) → override=True (will clobber) - dotenv_values() → hardcoded override=True (but doesn't modify os.environ, so it only affects interpolation)

Impact: If someone creates a DotEnv directly and calls set_as_environment_variables(), they get different default behavior than calling load_dotenv().

2. find_dotenv Can Crash with AssertionError

At main.py:367:

assert frame.f_back is not None

If the call stack has only one frame (e.g., in some embedded Python scenarios), this assertion fails with an unhelpful AssertionError. The function should handle this gracefully.

3. get CLI Command Returns Non-Zero Exit Code for Empty Values

In cli.py:144-148:

stored_value = values.get(key)
if stored_value:
    click.echo(stored_value)
else:
    sys.exit(1)

If a key exists but has an empty string value (e.g., EMPTY=), stored_value is "", which is falsy. The command exits with code 1 and prints nothing, making it indistinguishable from a missing key. This is a bug — it should check key in values instead.

4. set_key Creates Files Without Checking Directory Existence

set_key (main.py:193-245) calls rewrite which handles FileNotFoundError by creating an empty source (main.py:159-161). But if the directory doesn't exist, tempfile.NamedTemporaryFile will fail with FileNotFoundError, which is not caught. The error message will be cryptic.

5. DotEnv._dict Cache Is Never Invalidated

Once DotEnv.dict() is called, the result is cached in self._dict (main.py:77-78). If the underlying file changes between calls, the stale cache is returned. This is especially problematic for the get method (main.py:112-121) which calls self.dict() — calling get multiple times on the same DotEnv instance will always return the same results.

6. Variable Resolution Order Depends on override in resolve_variables

The resolve_variables function (main.py:289-311) builds a resolution environment where the order of precedence between os.environ and file-defined variables depends on override. This means the same .env file can produce different resolved values depending on the override flag, even when no environment variables are being modified. This is correct but surprising — dotenv_values(override=False) would resolve ${EXISTING_VAR} differently than dotenv_values(override=True).

7. _is_file_or_fifo Has a Race Condition

At main.py:474-482:

if os.path.isfile(path):
    return True
try:
    st = os.stat(path)
except (FileNotFoundError, OSError):
    return False
return stat.S_ISFIFO(st.st_mode)

Between the os.path.isfile check and os.stat, the file could be deleted or replaced. This is a TOCTOU (Time of Check, Time of Use) race condition. The same pattern exists in _get_stream which calls _is_file_or_fifo and then open() separately.

8. rewrite Preserves Mode Only for Regular Files

At main.py:150-155:

path_stat = os.lstat(path)
original_mode = (
    stat.S_IMODE(path_stat.st_mode)
    if stat.S_ISREG(path_stat.st_mode)
    else None
)

If the original file is a symlink (and follow_symlinks=False), os.lstat returns the symlink's metadata, but stat.S_ISREG checks the symlink itself (which is not a regular file), so original_mode is None and the mode is not preserved. The replacement file gets default permissions.