From legacy systems to microservices: Transforming auth architecture

What_Developers_Should_Know_about_composable_16.9.png

Contentstack receives billions of API requests daily, and every request must be validated to be a valid Contentstack identity. It is a common industry practice to achieve this using some sort of “identity token" for every request. Imagine having multiple types of identity tokens, such as session tokens, OAuth tokens, delivery tokens, management tokens, etc.

The problem of securing billions of API requests daily can be challenging. We decided to address this by spinning up a new team that handles the complex problems of user authentication and authorization in a token-agnostic platform.

Our transition journey

Contentstack started as an API-first headless CMS platform that allowed content managers to create and manage content while simultaneously and independently enabling developers to use Contentstack's delivery API to pull that content and render it to create websites and applications. This means that Contentstack’s traffic increases proportionately to the traffic received by our customers' websites and applications.

With increased traffic and usage, we catered to various new use cases by developing new features. These features were powered by a set of microservices, each catering to a particular feature domain and needing support for processing multiple identity tokens that had roles and permissions associated with them. The whole system had turned out to be quite complex, and performing auth had become a great challenge. This prompted us to redesign our auth architecture, which addressed the issues of being a token-agnostic and low-latency platform.

Read on to learn more about this journey and how we have been able to:

Transition from a monolith to a low latency microservices-based auth (authentication plus authorization) and rate-limiting architecture.
Set up centralized authentication for multiple (any domain) microservices that are part of the same Kubernetes cluster.
Set up decentralized and self-serviced, policy-based authorization for internal services and teams.

Increasing feature sets increased domain microservices, which increased the complexity of performing auth.

Monolithic auth architecture

Monolithic architectures can be difficult to maintain, scale and deploy. In a monolithic architecture, user authentication and authorization are typically tightly coupled with the application code, making it difficult to implement and maintain robust security measures. Monolithic architectures often rely on a single authentication and authorization mechanism for the entire application, which can limit the flexibility of the system to accommodate different types of users or access levels.

Performing auth in a typical monolithic architecture.

In monolithic architectures, the steps involved in auth are the following:

Users use their credentials at the client to generate a session token or use an existing identity token to generate other identity tokens.
Users then use the generated identity token to perform a business operation by making a request to the application server.
Once a request is received at the application server, the authentication middleware authenticates the token and forwards the request to the business module.
The business module performs the business operation based on the authorization rules applied to the user identity.

Problems with monolithic auth architecture:

Authentication and authorization logic is mixed with the business logic.
Changing the way an identity performs an operation on a resource involves a change in the associated auth-related logic.
Each domain individually implements the authorization logic, causing a difference in implementation.
Since authorization logic is deeply nested in business logic, we lack visibility into authorization rules applied to a resource.
Shipping of new authorization logic requires a fresh deployment of the application image.
New microservices require knowledge of various identity tokens and resource authorization rules to be applied.

Microservices auth architecture

Microservices offer a more flexible, modular approach that allows for easier maintenance, scalability and deployment. With microservices, each service can be developed, deployed and scaled independently, allowing for faster time-to-market, improved fault tolerance, and better alignment with modern development practices. Additionally, microservices offer more efficient use of resources and better support for diverse technology stacks.

Authentication

Why centralized authentication?

Centralized authentication is a security model in which a central authority manages authentication, such as a server or service, rather than it being distributed across multiple systems or applications. There are several reasons why centralized authentication is commonly used and considered advantageous, including increased security, simplified management, improved user experience and lower costs.

While there are some drawbacks to centralized authentication, such as the increased risk of a single point of failure and increased complexity in managing the central authority, the benefits often outweigh the risks.

Centralized authentication and rate-limiting at the edge of the service mesh.

The steps involved in the centralized authentication process are the following:

Any incoming request to the Kubernetes cluster first lands at the Istio ingress gateway.
The request containing the identity token is proxied to a central authentication gRPC service with the help of envoyproxy's external authorization filter.
The central authentication service queries Redis with the identity token and metadata associated with the request.
Redis responds with the identity associated with the token and the current rate-limit count based on the request metadata.
The central authentication service responds to Istio with either of the following:
Authenticated response with user context attached to the request in the form of request headers
Unauthenticated response
Ratelimit exceeded response
An authenticated request containing the user context is then forwarded to the upstream service.

Advantages over the monolithic architecture:

Easier to onboard newer microservices to central authentication service by using label based istio-injection.
All requests are authenticated and rate-limited at the edge of the service mesh, ensuring that each request entering the cluster is always rate-limited and authenticated.
The request forwarded to the upstream microservice has user identity context attached to it in the request headers, which can be further used for applying authorization rules.
Keeping centralized authentication eliminates the problem of multiple mutations performed by the upstream microservices on the identity of the token.

Authorization

Centralized authorization

We tried a model where along with authentication and rate limiting, we also added authorization as a responsibility of the central authentication and rate limiting service. The service would first identify the incoming request’s identity from the token and apply rate limiting based on the request metadata. Once the user identity is known, authorization rules could be applied to the user’s identity, thereby performing the entire Auth at the edge of the service mesh.

Problems with this model are the following:

This model could only perform basic authorization at the edge based on the request metadata provided, such as validating organizations, stacks, etc. However, it could not perform fine-grained authorization, such as finding out which content types the logged-in user had access to.
For RBAC, each domain has its roles and permissions associated with it; performing authorization for such requests requires knowledge of the upstream domain and leads to the addition of domain-specific logic in the centrally managed domain-agnostic platform.
With newer domain microservice additions, this again would lead to the problem of lacking visibility into authorization rules applied to a resource.

Distributed authorization with central authorization service

We then tried implementing a model where we distributed authorization to the upstream microservices where each upstream microservice makes a call to a central authorization service. The authorization service has access to all the roles and permissions of different domains and was able to give appropriate authorization results. Authorization could now be performed from the upstream service’s business module by making a network request using Kubernetes cluster networking to avoid making a call over the internet.

Problems with this model are the following:

The central authorization service becomes a single point of failure.
Any change in the API contract defined by the central authorization service requires all the upstream services to abide by it and makes shipping these changes independently a complex task.
Performing authorization adds a network hop, thereby increasing the latency.

Distributed authorization with the sidecar pattern

Learning from the previously discussed disadvantages, we wanted to build a model that had authorization distributed, low latency and made shipping authorization logic an independent activity.

Architecture

The architecture involves the following components:

Auth sidecar
Central policy service
Auth SDK

Architecture for authorizing an authenticated request with the sidecar pattern.

Auth sidecar

The auth sidecar is a gRPC service that gets injected along with the microservice’s application container in the same Kubernetes pod. Let’s understand how this architecture helped us tackle the previously mentioned problems.

Single point of failure: The auth sidecar service runs with the application container in the same pod, and any case of failure is only limited to the current pod. Restarting the pod gives us a fresh set of application and auth sidecar containers.

Independent delivery: Since the auth sidecar service container is shipped along with the application container, the application service can decide which version of the sidecar image to use, thereby making the delivery of newer versions of the authorization sidecar independent.

Low latency: There is no network hop involved in making a gRPC call to the auth sidecar running in the same pod. This helps the application to get the authorization result with very low latency (in a few milliseconds).

Updating authorization logic: The auth sidecar periodically downloads fresh policy bundles; any time there is a change in policy bundle coming from the central policy service, the auth sidecar updates its local policy cache with the new bundle.This way, updating authorization logic does not involve a fresh deployment/restart of the application container.

Components involved in auth sidecar

Responsibilities of the components involved in the authorization sidecar.

Aggregator: The responsibility of the aggregator is to fetch authorization-related data for the current identity based on the metadata provided by the application service in the gRPC call. It then aggregates it to be evaluated against the authorization policy.

OPA Engine: We use OPA (Open Policy Agent) to periodically download fresh policies and evaluate the policy path mentioned in the gRPC call against the aggregated data.

Central policy service

The central policy service is a repository of policy bundles (*.rego files) which are independently managed by the domain microservices. The maintainers of the domain microservices create these policies for various resources that need authorization. Since these policies only involve rules, it greatly increases the visibility of authorization rules being applied to a particular resource.

Auth SDK

The auth-sdk is an internal library that we developed that helps the developers of upstream microservices to easily communicate with different auth components. It can do the following:

Extract user identity and other useful information attached in the request headers by the central authentication service

Discover various auth components and streamline communicating with them

Expose different helper methods to perform any auth-related activity on behalf of the application service

Redesigned (new) architecture:

Tracing the request lifecycle in our redesigned auth architecture.

Conclusion

Microservices-based architectures can help address some of these challenges of monolithic architecture by separating user authentication and authorization into individual services, which can be developed, deployed and maintained independently. This approach can provide greater flexibility, scalability and security for user authentication and authorization.

However, it's important to note that transitioning to a microservices-based architecture can also come with some challenges, such as increased complexity and a need for more advanced DevOps practices. Proper planning, implementation and ongoing maintenance are crucial to ensuring a successful transition.

You may find interesting

Learn how to drive business forward and build better customer experiences.

How a Composable DXP drives experimentation, innovation and digital transformation

Today's era of relentless technological evolution is challenging companies to break free from the cookie-cutter molds of monolithic solutions and embrace technologies that harmonize their digital strategy with unrivaled agility.To keep pace with customer needs, businesses are no longer playing it safe but instead embracing experimentation and innovation to forge ahead in the race of digital transformation.Enter composable DXPs. A composable digital experience platform (DXP) can be the game-changer your marketing and creative content teams have been waiting for. As businesses strive to provide customers with personalized digital experiences across all channels, the need for a robust, scalable and adaptable technology stack becomes paramount.In this article, we'll explore what a composable DXP is, how it can revolutionize your approach to content and digital experiences, and the benefits it offers your business.What is a Composable DXPA composable DXP is a modern approach to building digital experiences. With a composable architecture, businesses can create, manage and deliver digital experiences across various channels and platforms by leveraging a range of interoperable, modular tools. The fundamental components of a composable DXP are:Headless CMS: A content management system (CMS) that allows content creation, easy updating, and management separately from the front-end layer your customers see.Composable technology: Imagine being able to build your content strategy from a set of building blocks that can be easily configured and reconfigured to work best for you. Composable technology enables businesses to use modular and lightweight components to build tailored digital solutions quickly.Composable architecture: A flexible and adaptable framework that supports the rapid integration of business-critical applications and tools, empowering teams to build better customer experiences.How to leverage a DXP for better contentWith an expansive catalog of configurations, applications, and diverse components, a composable DXP empowers marketing and creative teams to experiment with various combinations to achieve the desired results. This capability helps foster innovation and allows teams to respond quickly to changing customer needs. Here are five ways a composable DXP can drive better content:Personalized digital experiences: Leverage customer data, behavior analysis, and customer journey mapping to deliver highly-personalized, relevant experiences across all your digital channels.Omnichannel: Enable seamless content delivery and interactions across every device and touchpoint you rely on to reach your customers.Experimentation: Empower your teams to innovate and experiment with different tools, solutions, and content strategies, all from a single environment.Collaboration: Streamline collaboration between departments, teams, and individuals by using a centralized platform that fosters efficient workflows and automates manual tasks, enabling your teams to create faster.Scalability: Leverage the scalability of the DXP to quickly adapt to market changes, drive growth, and maintain your competitive edge.Benefits of a Composable DXPBeyond producing better content and creating more space for innovation, a Composable DXP offers businesses the digital tools to create meaningful customer experiences. Companies that migrate away from their rigid technologies and uplevel their digital capabilities have a full toolbelt for swiftly responding to market demands, improving customer satisfaction, increasing operational efficiency and maintaining a competitive edge in the market. How, you might ask?1. Speed-to-marketComposable DXPs enable businesses to respond quickly to changing market dynamics, customer needs, and technology advancements. By adopting a composable architecture, you can minimize time spent on complex integrations and cumbersome traditional CMS systems. As a result, your teams can build and launch new digital experiences much faster.2. FlexibilityA composable DXP provides the foundation for a flexible tech stack that can continuously evolve and adapt, as your business evolves. Businesses can choose best-of-breed technologies that align with their unique requirements, allowing them to build, experiment, and adjust their strategies in real time.3. Cost-efficiencyGone are the days of being locked into vendor agreements and limiting technologies. Businesses can select the tools they need according to budget and requirements. This facilitates better resource allocation, helping organizations operate in a leaner, more efficient manner.4. Innovation and experimentationA composable DXP fosters a culture of innovation by empowering marketing and creative teams to experiment and iterate. This leads to accelerated digital transformation and triggers an innovation domino effect across the organization.Grow faster, scale better, experience more ROIA composable DXP can be a crucial driver of experimentation, innovation and digital transformation for businesses. By leveraging a headless CMS and composable architecture, businesses can unlock the true potential of their digital ecosystem, experience greater ROI from increased customer loyalty and, ultimately, stay ahead of the curve by delivering exceptional digital experiences that drive growth and success.About the authorMarina Rusinow is a Senior Manager of Solutions Marketing at Contentstack with over a decade of product marketing experience.Learn more about Marina

Read

Building a scalable DXP - Part 1

Scale is one of the most common and fundamental problems digital teams have to confront in order to grow their businesses. But more than just a key to unlocking growth, how a team thinks about scale can affect costs, ongoing resource capacity, feature roadmaps, and even an organization’s overall pace of innovation. In the universe of digital experience (DX), which spans all the way from the devices at the frontier of your audience’s senses all the way to the cloud and hardware that powers your company’s back-of-house operations, scale applies in some way to nearly every facet between these two ends. Understanding how the systems, people and processes that make up this end-to-end DX assembly line fit together and affect each other is key to unlocking that scale.This article aims to explain how to scale your digital experience platform (DXP). In it, we will discuss the meaning of scale, how to approach scale within your organization, and why it’s important. What exactly does it mean to scale?Simply put, scale is the relationship between the inputs and outputs of a given system. When people refer to scaling a business, they are typically talking about increasing outputs, like sales and revenue, while holding constant or decreasing inputs, like costs and materials. In an engineering-oriented team, scale often refers to hardware and software, such as the ability to flex or expand capacity to process more operations. If you work on a content or marketing team, scale usually refers to growing your audience, increasing engagement, and publishing more and better content. On the business and leadership teams, scale usually means increasing revenue.But these definitions of scale are oversimplified because they focus only on the outputs of these systems while ignoring the inputs. Or to put it another way, when we are talking only about increasing the outputs of a system, we’re talking about growth. When we talk about increasing outputs relative to the inputs, we are talking about scale.A “scalable” case studyConsider a digital agency as a hypothetical case study. The agency builds highly engaging promotional websites, and its revenue is directly tied to the number of websites it can implement for its customers. Let’s assume the agency is high touch and that each website takes a total of 1,000 hours to successfully design, build and launch. Let’s also assume that the agency employs 10 people, for a total of 20,000 available hours per year (ignoring PTO) and enough resources and the proper staffing configuration to successfully sell and deploy a maximum of 20 new sites per year.A blunt approach to scaling this business model would be to demand more than 20 sites out of the talent. In this approach, leadership sees the staff as the enemies of scale and demands more output from them in the form of working faster and staying late to meet aggressive goals. Indeed many companies attempt this strategy at the cost of talent attrition, sloppy work, constant training of new employees, missed deadlines and lost customers. Launching just two additional sites per year would require every employee to work an average of about one extra half-day every week of the year, leaving scant time for family as well as the occasional urgent customer support request.Another approach is hiring additional talent and slowly scaling up output while demand catches up to the newly increased capacity. Although this approach is common, it comes with the risk of increasing investment before increasing revenue, and when applied responsibly it can take some time to pay off. Leaders who choose this tactic will hire, say, two more staff with the aim of selling and launching about four additional sites per year. While this approach is intuitive, once the full cost of two FTEs, the opportunity costs of training them, and the revenue from four additional sites are factored into the balance sheet, it’s easy to see that this approach could be slow to take off and perhaps actually decrease the agency’s profit margin over time if it doesn’t go well.Now consider a third approach, where leadership recommends that each team looks for opportunities to repurpose generic work developed for other customers in order to reduce the number of hours required to implement new websites. Over the course of building the next few sites, the teams are able to develop reusable components of their work, for example, in the form of code and design libraries for commonly requested solutions, which in turn results in an average reduction of 200 hours across all website deliverables. This means that the agency can now produce 20 sites of the same or better quality in only 16,000 hours, which increases output capacity to 25 websites in the same original 20,000 available hours without adding staff or increasing costs.Scale smarter, not harderWhile each of these three examples are strategies for how an agency might approach scaling up output, the third example is a clear frontrunner because it has the highest impact on increasing output while at the same time holding inputs relatively constant. The third approach is also the only example out of the three that can lead to economies of scale. Economies of scale describe the state of a relationship between an input, such as cost per unit, and an output, say, the number of units produced, where the cost per unit actually goes down as you create more units. This relationship between inputs and outputs is what people typically mean when they say they want to scale something. It is not just that they want outputs to grow, but that they want outputs to grow and the cost of inputs to stay flat or decrease at the same time.Although this case study centers around an agency and looks at three simplified approaches to scaling the overall business outputs, it’s useful to shed light on the fundamental concepts of scale before unpacking the specifics of scaling a DXP. Here we’ve highlighted two essential tenets that will benefit digital leaders as they work to scale nearly any system: There are many ways to approach a problem of scale, some sustainable and some unsustainable. Scaling on the backs of employees, for instance, is not a sustainable approach. There is nothing wrong with hustling to achieve a goal, but hustle alone lacks the strategic thinking that unlocks scale, which leads us to our second tenet.Effective and durable scaling centers around creativity and is founded upon understanding — not brute force. But don’t be discouraged if you feel you or your team lacks creativity or your processes are too complex for anyone to understand. As we will see in the following articles, there are some tried and true methods for creating the conditions for creativity to emerge, and if you apply them with your teams you will have a higher likelihood of successfully achieving scale, no matter what it is you wish to scale.What’s next? Now that we’ve established a baseline understanding of scale, in Part 2 we will set the agency example aside and explore specifically what it means to scale a DXP. Stay tuned!About the authorDean Haddock is a Senior Product Manager at Contentstack who focuses on platform, hosting and data engineering.Learn more about Dean

Read

Why MACH architecture matters

Have you been hearing about MACH lately? It’s currently one of the biggest buzzwords out there — and for good reason. The acronym has spread like wildfire throughout the digital world. Endorsed by analysts and IT experts; frowned upon by not-quite-as-tech-savvy business folks; initially belittled, now heavily embraced by the monoliths who are trying to MACH-wash their offerings and avoid extinction. But what exactly is MACH architecture? How does it compare to monolithic platforms? And where does composable fit into all of this? If you’ve been wondering lately, keep reading. You’re about to get the inside scoop. What is MACH architecture?MACH stands for Microservices, API-first, Cloud-native, and Headless. Let’s look at each individual term. Microservices: Microservices bring together loosely coupled, independently deployable small components or "services" to compose a more integrated application. They are exposed via well-defined APIs as the communication method between frontends and backends. They deliver faster responses, are more reliable, and can be deployed more frequently. API-first: Through APIs, best-of-breed components can be combined into a custom application built for specific business needs. Cloud-native: Being cloud-native in MACH architecture means leveraging cloud-based infrastructure and services, resulting in significant cost advantages by avoiding the high costs of maintaining on-premises infrastructure, such as hosting, site acceleration, security and uptime. Headless: Headless describes the decoupling of the front end from the back end. The front end can be anything customer-facing, from a web shop to social media to mobile apps. The backend is the layer where all the systems, processes and tools run to handle operations, including product information management, checkout and more.Headless vs composable vs MACHHeadless is all about decoupling the front end and the back end so they can operate independently and achieve higher flexibility and agility. Nowadays, you see many legacy platforms claiming to be headless when really all they offer is a few APIs. Don’t be fooled. If a monolithic, non-microservices-based architecture is lurking behind the scenes, you won’t be able to unlock the freedom that a true MACH architecture has to offer.Composable is yet another term that is being thrown around, many times used interchangeably with MACH. They are not the same though. Composable is a modular development approach and enables brands to “compose” unique customer experiences by plugging best-of-breed building blocks like cart, checkout and payments into their technology stack. Those components, though, are not required to be MACH-compliant.Assembling a composable architecture using MACH-compliant building blocks is the only way to truly embrace the power of composable technology. Essentially, composable takes a step beyond headless. It breaks down the entire platform into individual components that can be independently plugged in, customized and even replaced. That allows companies to integrate curated best-of-breed components to tailor their technology stack and, consequently, the customer experience. What are the benefits of MACH technology?MACH-based architecture provides the technical backbone to make businesses future-fit. It creates an environment in which every component is pluggable, scalable, replaceable and can be continuously improved through agile development to meet the ever-evolving business requirements.When combined, the technologies that provide the foundation of MACH architecture provide remarkable advantages:Flexibility: Decoupling of frontend and backend allows for more agile development and deployment.Scalability: Auto-scaling ensures seamless handling of traffic peaks with zero downtime. No more Black Friday outages!Speed: Allows for faster iteration and testing, and the cloud provides instant provisioning and deployment of resources.Lower costs: Reduces the need for costly and frequent investments in hardware, software and infrastructure.Resilience: Reduces the risk of application downtime; has built-in redundancy and failover mechanisms.If you want to go further, faster, a MACH architecture will get you there.MACH architecture vs. monolithsThe time of the monolithic giants — think the likes of SAP, Oracle and Salesforce — is coming to an end. No monolithic platform can deliver the same speed, functionality and performance that a MACH architecture can deliver. What may seem like a blessing when you sign on with a monolith (you only buy once, with one contract and one point of contact) quickly becomes a curse if you want anything but the standard procedure. Thinking about an innovative, unique customer experience? You’ll need to beg the monolith to create it for you, and chances are slim they will unless you’re the biggest customer in their portfolio. With a MACH architecture, you are in the driver’s seat. You’ve got a brilliant idea for a thrilling new customer experience? Plug and play the best-of-breed solutions and watch your business thrive. With a monolith? Wait until the trend has passed to even get a response to your request. Is your business ready for MACH?Now that I’ve gotten you curious about the whole MACH thing, how do you know if MACH is right for your business? Start with an assessment of your current systems, processes and goals. Then, check the statements below. If you agree with any or all of them, it is time to make a move. Your current system is holding you back because it is too inflexible and slow to move at the speed necessary to keep up with shifting customer expectations and industry trends.You're looking to decrease TCO and increase ROI as you are currently paying for resources you don't need.You want to improve customer experiences, but you are struggling to provide cohesive and personalized omnichannel experiences.You want to stay ahead of the competition by staying agile and responsive to market demands, but you're finding you can't keep up with your competitors.The bottom lineIn today’s uncertain market conditions, the only constant is change. Moving to a MACH architecture will not only make your business more agile and adaptable, it also reduces the total cost of ownership (TCO) often associated with outdated pricing and licensing models employed by the monoliths.A modern tech stack is key to making your business future fit. Make the move from monolith to MACH. Tailor your tech stack to your individual business needs and create that unique customer experience that will put you ahead of your competition — time and again.About the authorJasmin Guthmann is the Head of Corporate Communication at Contentstack and Vice President of the MACH Alliance.Learn more about Jasmin

Read

Join the conversation

Community

Join the Contentstack Community to find more answers and help from experts and users.

Join

For executives

Leverage composable tech to drive business forward

Learn

For digital leaders

Learn how to deliver better digital experiences, faster

Learn

For developers

Learn how to build better digital experiences

Learn