Cloud Architecture

The moclojer is an Open Source product. To serve it as a service, we created an event-based architecture for event propagation and data processing, generating triggers for specific actions.

Backend Services

The following are the services and integrations that keep moclojer’s backend alive:

back/api: acts primarily as an API gateway for all client interactions, handling requests and routing them to appropriate services through message queues. Encapsulates initial business logic for user mocks, propagating necessary events to other services;
yaml/generator: updates and validates every mock for events like creation, deletion, and updates. It manages the unified mock, aggregating all user mocks for use in moclojer-foss’s runtime. Generated files are stored in our object-store;
cloud/ops: manages and monitors our infrastructure, user mock domains and DNS records, by applying a facade on our two current used cloud solutions, namely DigitalOcean and CloudFlare;
moclojer/foss: the open source version of moclojer, extended as a framework within an isolated service. Based on the content of the unified mock given from consequent events from yaml/generator, serves unique mock endpoints, acting therefore as the final API.

Web Frontend

We serve a web interface for user interactions. This interface is responsible for interacting with our Backend Services and deliver updates to our user.

Stack: ClojureScript, Helix, Refx, TailwindCSS.

Data Layer

Postgres: we use postgres as our main database.
Redis: since moclojer is event-driven, we use Redis as a message broker, through its Pub/Sub mechanism.

Communication and Integration

p001/proxy: a reverse proxy for our internal services that accesses outsider APIs;

Infrastructure

We use DigitalOcean as our infrastructure provider, our goal is to serve moclojer foss in a scalable way for all users. To achieve this, we use the App Platform to serve applications, Managed Database to provide managed databases (PostgreSQL and Redis), and Droplet to serve the reverse proxy and Supabase for frontend authentication.

Application logs are sent to Logtail, installed via DigitalOcean add-ons.

App Platform

The App Platform is a platform as a service (PaaS) that enables developers to build, deploy, and scale applications quickly and easily. It supports several programming languages, including Clojure 💜, and provides a range of features to help developers build and deploy applications more efficiently.

We have two applications on the App Platform:

moclojer-app

static_sites serving the frontend, with ingress (public access)
services serving the back/api, with ingress (public access)
workers running the yaml/generator and cloud/ops
databases running the postgres and redis/mq managed

moclojer-foss

services serving the moclojer/foss, with ingress (public access) - with support for multiple domains

We use managed infrastructure to focus our efforts on the product we serve as a service, not on the infrastructure that supports it. DigitalOcean helps us stay focused on what really matters.

Overall Architecture

This is a simplified overview of how our described services connect and interact between themselves after built, deployed and running on Digital Ocean.

graph TD;

    subgraph "DigitalOcean"
        subgraph "database services"
            postgres
            redis/mq
        end

        subgraph "App Platform"
            back/api
            frontend
            cloud/ops
            yaml/generator
            moclojer/foss
        end

        subgraph "Droplet"
            p001/proxy
        end

        paas-service
        object-store
    end

    back/api --> postgres;
    back/api --> redis/mq;
    back/api --> object-store;
    frontend --> back/api;
    frontend --> supabase/auth;
    cloud/ops --> redis/mq;
    cloud/ops --> p001/proxy --> dns-services;
    cloud/ops --> paas-service;
    yaml/generator --> object-store;
    moclojer/foss --> object-store;

Overall Sequence Behaviour

This is an overview of how an optimistic run of the use case of a mock creation gets our services to work together.

sequenceDiagram;

    participant frontend;
    participant back/api;
    participant db;
    participant yaml/gen;
    participant cloud/ops;
    box cloud providers
        participant cloudflare;
        participant digitalocean;
    end

    frontend ->>+ back/api: post /mocks;
    back/api ->>+ db: insert mock;
    db ->>- back/api: ok;
    back/api -->> yaml/gen: generate mock file;
    back/api ->>- frontend: 200 ok;

    loop mock offline
        frontend ->>+ back/api: get /mocks/123/publication;
        back/api ->>+ db: select mock publication status;
        db ->>- back/api: ok;
    end

    yaml/gen -->> yaml/gen: generated unified mock file;
    yaml/gen -->> cloud/ops: create domain;
    cloud/ops ->>+ cloudflare: create DNS record;
    cloudflare ->>- cloud/ops: 200 ok;
    cloud/ops ->>+ digitalocean: create domain;
    digitalocean ->>- cloud/ops: 200 ok;
    cloud/ops ->> back/api: mock published;
    back/api ->>+ db: update mock published;
    db ->>- back/api: ok

Event Flow

As previously mentioned, we use Redis MQ to facilitate communication between services via Redis’s Pub/Sub capabilities. Our queues are, separated by service, specifically:

mock/api

domains.verification.dispatch: a job at every 2 minutes that aggregates offline mocks to be verified, dispatching them to domain.verify;
unified.verification.dispatch: a job at every 5 minutes that aggregates published mocks to be verified, dispatching them to unified.verify;
mock.publication: sets a mock publication status;

yaml/generator

mock.changed: received when a mock content changes. It is saved to our object-store, and packaged to be unified;
mock.unified: aggregates the new changed mock to the global unified mock, used thereafter by moclojer/foss;
mock.deleted: removes given mock from our object-store and signals the recreation of the global unified mock;
unified.verify: given the aggregated mocks, verifies if each of them is in teh unified mock, signaling the recreation if not;

cloud/ops

domain.create: interacts with DigitalOcean and CloudFlare to set up the new domain host and DNS records;
domain.verify: verifies the existence and validity of the newly created host, firing mock.publication thereafter;

moclojer/foss

restart.mocks: syncs to the unified mock, restarting the running moclojer server instance;