[Pelis Agent Factory Advisor] Pelis Agent Factory Advisor - Agentic Workflow Opportunities for gh-aw-firewall (2026-03-04)

[github-actions[bot]]

📊 Executive Summary

The gh-aw-firewall repository has a strong and security-oriented agentic workflow foundation — particularly impressive given its domain as a security/firewall tool. With 28 compiled lock files and ~17 agentic .md workflows, the repository already operates at maturity level 3.5/5. The most impactful gaps are: (1) no automated issue triage/labeling, (2) no meta-monitoring of the agentic workflows themselves, and (3) missing continuous code-quality agents (simplifier, breaking-change checker). Addressing these would push maturity to 4.5/5 with relatively low effort.

---

🎓 Patterns Learned from Pelis Agent Factory

Key patterns from the Peli's Agent Factory blog series and the githubnext/agentics repository:

Pattern	Description	Relevance to this Repo
Specialization over monoliths	Many focused agents beat one general-purpose agent	✅ Well-practiced here
Issue triage as "hello world"	Every mature repo has auto-labeling	❌ Missing
Meta-agents watch other agents	Workflow Health Manager / Audit Workflows create the most value in large factories	❌ Missing
Causal chain automation	Agents create discussions → issues → PRs in sequence	✅ Partially practiced
Security agents for security tools	Daily Malicious Code Scan, Static Analysis Report	🟡 Partial
Continuous simplicity	Daily code simplifier cleans up after fast development	❌ Missing
skip-if-match guards	Prevent duplicate in-flight work	✅ Used in most workflows
Cache memory	Cross-run state for deduplication and trend analysis	✅ Used in issue-duplication-detector, security-review
Multi-engine smoke tests	Validate core functionality with multiple AI engines	✅ Excellent coverage

Compared to the factory, this repo has disproportionately strong security coverage (expected for a firewall tool!) but is weak on code quality agents and meta-monitoring, which are the highest-ROI additions for a complex, growing TypeScript codebase.

---

📋 Current Agentic Workflow Inventory

Workflow	Purpose	Trigger	Assessment
secret-digger-claude/codex/copilot	Red-team: search for secrets in container	Hourly	✅ Excellent — unique to this repo
smoke-claude/codex/copilot/chroot	Engine smoke tests	Every 12h + PR	✅ Excellent — multi-engine
build-test-{8 langs}	Build/test firewall against 8 language stacks	PR + schedule	✅ Comprehensive
security-review.md	Daily security review & threat modeling	Daily	✅ Strong
security-guard.md	PR security review	Every PR	✅ Valuable
dependency-security-monitor.md	CVE monitoring + patch PRs	Daily	✅ Good
ci-doctor.md	Investigates CI failures	workflow_run failure	✅ Good
ci-cd-gaps-assessment.md	Pipeline gap analysis	Daily	✅ Useful
test-coverage-improver.md	Weekly coverage PRs on security code paths	Weekly	✅ Well-scoped
doc-maintainer.md	Sync docs with code changes	Daily	✅ Good
cli-flag-consistency-checker.md	CLI flags vs docs consistency	Weekly	✅ Domain-specific gem
issue-monster.md	Task dispatcher to Copilot coding agent	Issue opened + hourly	✅ Good
issue-duplication-detector.md	Duplicate issue detection (with cache)	Issue opened	✅ Good
plan.md	/plan slash command	/plan comment	✅ Useful ChatOps
update-release-notes.md	Enhance release notes on publish	Release published	🟡 Narrow scope
pelis-agent-factory-advisor.md	This workflow	Daily	✅ Meta-awareness

---

🚀 Actionable Recommendations

⚡ P0 - Implement Immediately

P0.1: Issue Triage Agent

What: Automatically label new issues with standard categories and leave a welcoming comment explaining the label.

Why: Currently, issues open with no labels. This creates friction for maintainers triaging work. The Issue Monster already picks up issues, but it dispatches them without categorization. Given the security-sensitive nature of this repo, proper labeling (e.g., bug, security, documentation, enhancement, question) helps prioritize triage.

How: Add a simple workflow triggered on issues: [opened, reopened] that analyzes issue content and applies one label from an allowed list. Use lockdown: false since this is a public repo and you want to process issues from all contributors.

Effort: Low (< 2 hours including compile)

Example frontmatter:

---
name: Issue Triage
on:
  issues:
    types: [opened, reopened]
permissions:
  issues: read
tools:
  github:
    toolsets: [issues, labels]
    lockdown: false
safe-outputs:
  add-labels:
    allowed: [bug, security, documentation, enhancement, question, help-wanted, good-first-issue, ci-failure]
  add-comment: {}
timeout-minutes: 5
---

---

P0.2: Workflow Health Manager / Audit Agent

What: A daily meta-agent that audits all other agentic workflow runs, tracking success rates, costs, failures, and anomalies. Reports findings as a discussion.

Why: With 17+ active agentic workflows running multiple times daily (the secret diggers alone run hourly × 3 engines!), there's no automated monitoring of the monitoring. The ci-doctor.md handles failures but doesn't provide holistic health analytics. Portfolio Analyst in the factory identified workflows costing unnecessary tokens. This repo is approaching that scale.

How: Use the agentic-workflows tool + GitHub Actions API to analyze recent run history. Track per-workflow success rates, token usage trends, runs skipped vs executed, and anomalous patterns (e.g., a workflow that suddenly starts failing or timing out).

Effort: Medium (uses existing agentic-workflows tool, primarily configuration)

Source to adapt: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/audit-workflows.md

---

🎯 P1 - Plan for Near-Term

P1.1: Breaking Change Checker

What: On each PR, detect if the change introduces backward-incompatible changes to the public CLI interface, Docker API surface, or configuration format.

Why: This is a security-critical firewall tool used by other agents. Breaking changes to the CLI flags, docker-compose config format, or squid.conf generation could silently under-protect agents running the old version. The CI/CD gaps assessment workflow already identifies this as a gap. The Pelis factory saw this catch incidents before production.

How: PR-triggered workflow that compares src/cli.ts argument definitions, src/types.ts interfaces, and container entrypoint scripts to the previous version. Creates an alert issue for breaking changes.

Effort: Medium

Source to adapt: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/breaking-change-checker.md

---

P1.2: Automatic Code Simplifier

What: Daily workflow that reviews recently modified TypeScript, shell scripts, and container configs for simplification opportunities, then opens PRs.

Why: This codebase has grown significantly (1500+ line docker-manager.ts, complex setup-iptables.sh, multi-provider API proxy logic). The Pelis factory's Code Simplifier achieved 83% merge rate on PRs. TypeScript is particularly well-suited for this — there are patterns like nested conditionals, repeated error handling, and manual type assertions that can be cleaned up continuously.

How: Daily trigger, analyzes commits from last 3 days in src/, containers/, scripts/ci/. Creates PRs with simplifications. Uses skip-if-match to prevent pile-up.

Effort: Low-Medium

Source to adapt: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/code-simplifier.md

---

P1.3: Changeset Generator (Release Automation)

What: When a PR is merged, or on a schedule, analyze commits since last release and generate a comprehensive changelog entry + version bump proposal.

Why: The existing update-release-notes.md only kicks in after a release is published, and only enhances the description. There's no workflow that proactively identifies "we've accumulated enough changes for a v0.X.0 release" or automates semantic versioning decisions. The factory's Changeset workflow achieved 78% merge rate on 28 proposed PRs.

How: Weekly workflow that compares commits since last tag, determines major/minor/patch based on conventional commit types, and opens a PR updating CHANGELOG.md and package.json version.

Effort: Medium

Source to adapt: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/changeset.md

---

P1.4: Daily Malicious Code Scan

What: Daily review of recent code changes for suspicious patterns — especially important for a security tool that handles iptables rules, network routing, and container configurations.

Why: This repo is itself part of the security infrastructure for AI agents. Supply chain attacks targeting firewall/proxy tools are high-value targets. The factory's malicious code scan provides an automated defense layer. Given that this tool processes --allow-domains inputs that get injected into squid.conf and iptables rules, any code changes that weaken input validation deserve extra scrutiny.

How: Daily workflow examining commits from last 24h in containers/agent/, src/squid-config.ts, src/host-iptables.ts. Flags anything suspicious for human review.

Effort: Low (template adaptation)

Source to adapt: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/daily-malicious-code-scan.md

---

📌 P2 - Consider for Roadmap

P2.1: Schema/Contract Consistency Checker

What: Weekly check that CLI flags in src/cli.ts, the TypeScript types in src/types.ts, docker-manager.ts config generation, and all documentation stay synchronized.

Why: The existing cli-flag-consistency-checker.md does something similar but is focused on flag documentation. A broader schema consistency checker would also verify that runtime config generation matches type definitions — for example, that every WrapperConfig field in types.ts is actually documented in docs/environment.md and handled in docker-manager.ts. The factory's Schema Consistency Checker caught schema drift that took days to notice manually.

Effort: Medium

---

P2.2: Documentation Noob Tester

What: Periodically test the docs-site (Astro Starlight at docs-site/) as a first-time user would — following the quickstart guide, checking that examples work, verifying links, and flagging confusing steps.

Why: AWF has a learning curve (iptables, Docker, Squid, multiple engine options). The docs-site is growing. New users are the most vulnerable to outdated examples. The factory version achieved 43% merge rate but 100% of flagged issues were legitimate concerns.

Effort: Medium (requires Playwright setup)

---

P2.3: Mergefest / Auto-Merge Main to PRs

What: Automatically merge main into long-lived PR branches to prevent merge conflicts accumulating.

Why: With many agentic workflows creating PRs (test-coverage-improver, doc-maintainer, dependency-security-monitor), PRs can fall behind main quickly. The factory's Mergefest eliminates the "please merge main" ceremony.

Effort: Low

Source to adapt: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/mergefest.md

---

💡 P3 - Future Ideas

P3.1: Portfolio Analyst

What: Weekly analysis of all agentic workflow costs and token usage, identifying optimization opportunities.

Why: The secret diggers run hourly × 3 engines = 72 runs/day. Understanding their token usage and whether they're finding anything is important for cost management.

Effort: Medium

Source: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/portfolio-analyst.md

---

P3.2: Integration Test Result Trend Analyzer

What: Track integration test pass/fail rates over time across the 8 language build tests (Go, Rust, Java, .NET, Node, C++, Deno, Bun) and smoke tests, creating alerts when a language's success rate degrades.

Why: Currently each test run is investigated in isolation by CI Doctor. But a pattern where, say, the Java build tests start failing more often would indicate a systemic issue worth investigating proactively.

Effort: Medium (requires cache-memory for trend state)

---

P3.3: Docs Site Link & Freshness Auditor

What: Weekly check of docs-site/ Astro Starlight content for broken links, outdated version references, and code examples that no longer match the implementation.

Why: The docs-site is a first-class citizen of this repo with a technical writer custom agent. But freshness drift is inevitable and a dedicated auditor would catch it automatically.

Effort: Low

---

📈 Maturity Assessment

Dimension	Current	Target	Gap
Security automation	5/5	5/5	✅ None
CI/CD health	4/5	4.5/5	Add breaking-change checker
Documentation	3.5/5	4.5/5	Add noob-tester, docs auditor
Code quality	2/5	4/5	Add simplifier, schema checker
Issue management	3/5	4.5/5	Add triage, mergefest
Release automation	2.5/5	4/5	Add changeset generator
Meta-monitoring	2/5	4/5	Add audit/health manager

Current Level: 3.5/5 — "Advanced Practitioner". Strong specialized coverage (security is exceptional), but missing foundational patterns from the factory.

Target Level: 4.5/5 — "Highly Automated Factory". Implementing P0+P1 recommendations achieves this level.

What's needed: The biggest gaps are meta-monitoring (watching the watchers) and code quality automation. The P0 and P1 items above address all major gaps.

---

🔄 Comparison with Pelis Factory Best Practices

What this repo does well ✅

• Security-first mindset: The secret-digger triple (Claude/Codex/Copilot) running hourly is unique and impressive — more security coverage than the original factory
• Multi-engine smoke testing: Testing with 3 engines + chroot mode shows commitment to comprehensive coverage
• Domain-specific CLI checker: The cli-flag-consistency-checker.md is a beautifully tailored workflow specific to this tool's risk profile
• Skip-if-match discipline: All workflows properly prevent duplicate in-flight runs
• Cache memory usage: Both issue-duplication-detector and security-review use cross-run memory correctly
• Language matrix testing: Build-testing against 8 languages is beyond what most repos do

What it could improve 🔧

• Issue hygiene: No auto-labeling means issues accumulate with no categorization signal
• Meta-monitoring: No "meta-agent" watching the health of the 17+ agents
• Continuous code quality: Missing the virtuous cycle of simplifier → PR → merge that the factory uses to keep codebases clean
• Release lifecycle: Changeset automation is the missing link between daily work and versioned releases

Unique opportunity: Security-aware code quality 🔐

Given this is a firewall tool, the code simplifier and schema consistency checker could be tuned specifically for security-sensitive patterns: ensuring no regex becomes .*/.* in ACL rules, ensuring capability drop still happens before exec, ensuring iptables REJECT rules aren't accidentally reordered. This is a more powerful version of these workflows than the generic factory provides — security-aware code quality automation unique to this domain.

---

Analysis based on Pelis Agent Factory documentation at https://github.github.io/gh-aw/blog/2026-01-12-welcome-to-pelis-agent-factory/ and githubnext/agentics reference implementations. Cache notes updated at /tmp/gh-aw/cache-memory/advisor-notes.md.

AI generated by Pelis Agent Factory Advisor

• expires on Mar 11, 2026, 3:22 AM UTC