The Identity Trap: Why We Killed Our Custom IdP to Save Our Architecture

Introduction#

There is a seductive moment in every senior engineer's life where they look at the landscape of OAuth2, OIDC, and SAML providers and think: "I could build a lighter version of this in a weekend."

For mykb-auth, that is exactly what we did. We built a fully functional Identity Provider (IdP). It handled user registration, secure password hashing, session management, and JWT issuance. It was fast, lightweight, and we owned every line of code.

And then, despite it working perfectly, we made the decision to deprecate it entirely in favor of Zitadel.

This isn't a story about failing to build authentication. It is a story about recognizing the difference between "Commodity Infrastructure" and "Secret Sauce."

The Insight: Building your own IdP is like writing your own Linux kernel. You can do it, and you might learn a lot, but you will spend the rest of your life patching security vulnerabilities instead of building your product.

The Prototype: The Illusion of Simplicity#

We started with a clear goal: A Zero-Trust architecture for our AI-native knowledge base. We needed a system that could handle Identity (Who are you?) and detailed Policy (What can you do?).

In three days, we spun up a custom service using FastAPI and SQLAlchemy. We implemented:

AuthN: Bcrypt password hashing and JWT generation.
AuthZ: A custom Policy Decision Point (PDP) for ABAC (Attribute-Based Access Control).
Key Rotation: An internal mechanism for rotating signing keys.

It felt like a win. We had eliminated the "bloat" of external providers. But as we looked at the roadmap for "Phase 2," the reality of maintaining an IdP set in.

The "Build" Trap#

The code we wrote wasn't buggy. In fact, it was architecturally sound. The problem wasn't the code we had written; it was the code we hadn't written yet.

To make our custom IdP truly enterprise-ready, we were staring down the barrel of implementing:

MFA & Passkeys: Essential for security, complex to implement correctly.
SCIM Provisioning: Required for any enterprise integration.
Social Logins: Maintaining connectors for Google, GitHub, and Microsoft.
Security Patching: protecting against timing attacks and edge-case vulnerabilities forever.

"We successfully built a passport office. But we realized our business was about building the secure facility behind the gates, not printing the ID cards."

The Pivot: Decoupling AuthN from AuthZ#

We realized we were conflating two distinct problems: Authentication and Authorization.

Authentication (AuthN) is a Commodity. Verifying a user's identity via passwords or MFA is a solved problem. It is high-risk and low-reward.
Authorization (AuthZ) is our Domain. Deciding if User A can see Document B based on Tenant C's rules is unique to our business logic.

The New Architecture#

We moved to a hybrid model. We replaced our custom IdP with Zitadel to handle the commodity work of Identity. However, we kept our custom PDP (Policy Decision Point) and PEP (Policy Enforcement Point).

This allows Zitadel to issue the "Passport" (the JWT), while our internal services enforce the "Law" (Access Control).

Architecture diagram showing Zitadel issuing a token, the Gateway verifying it, and the PDP injecting headers

The Code: Zero-Trust Header Injection#

The beauty of this shift is how it simplified our downstream services. Our Gateway (PEP) now verifies the Zitadel token, consults our internal PDP, and injects validated headers.

The internal microservices no longer worry about OAuth scopes or token parsing. They simply trust the headers from the Gateway:

@app.get("/api/documents")
async def get_documents(request: Request):
    # The Gateway has already authenticated the user and 
    # calculated the ABAC filter. We just enforce it.
    
    # 1. Identity (from Zitadel via Gateway)
    user_id = request.headers.get("X-User-ID")
    
    # 2. Policy (from our Custom PDP)
    abac_filter_str = request.headers.get("X-ABAC-Filter")
    abac_filter = json.loads(abac_filter_str)
 
    # 3. Execution (Zero-Trust)
    # We pass the mandated filter directly to the database
    return await db.find(abac_filter)

Conclusion#

Dropping our custom IdP wasn't an admission of defeat; it was an act of maturity.

By adopting Zitadel, we instantly gained MFA, audit trails, and security compliance. By keeping our custom PDP, we maintained the flexibility to define complex, AI-driven access rules.

We stopped building infrastructure that already exists so we could focus on the logic that sets us apart. In the end, the best code you write is often the code you decide to delete.