LeetLLM
EasyObservabilityPython 3

Command Log Classifier

Summarize command outcomes from an agent run while preserving safe review logs.

25m3 sample tests6 hidden tests

Implement classify_command_log(events), a helper that summarizes shell commands from an agent run.

Requirements

  • Each event is a dictionary with cmd, exit_code, and optional stdout / stderr (missing streams are empty).
  • Return a dictionary with status, total, failed, first_failure_cmd, and redacted_logs.
  • status is "passed" when no command failed, otherwise "failed".
  • A command fails when exit_code != 0.
  • first_failure_cmd is the first failing command, or None.
  • redacted_logs is a list of strings built from stdout then stderr for each event in order.
  • Split each stream with newline rules equivalent to str.splitlines(); emit one entry per resulting line (including blank internal lines).
  • Each entry uses the format "{stream}: {line}" (for example "stdout: one", "stderr: err [REDACTED]"). Empty or missing streams contribute no entries.
  • Redact secret tokens matching the regex sk-[A-Za-z0-9_-]+ by replacing each match with [REDACTED] (token-level replace, not whole-line wipe).

Example

python
1events = [ 2 {"cmd": "pytest", "exit_code": 1, "stdout": "", "stderr": "failed"}, 3 {"cmd": "print", "exit_code": 0, "stdout": "sk-live", "stderr": "err sk-test_2"}, 4] 5summary = classify_command_log(events) 6assert summary["status"] == "failed" 7assert summary["total"] == 2 8assert summary["failed"] == 1 9assert summary["first_failure_cmd"] == "pytest" 10assert summary["redacted_logs"] == [ 11 "stderr: failed", 12 "stdout: [REDACTED]", 13 "stderr: err [REDACTED]", 14]

Constraints

  • Preserve log line order.
  • Leave inputs untouched.

Editor