How To Build A Real-Time App With GraphQL Subscriptions On Postgres

Original Source: https://www.smashingmagazine.com/2018/12/real-time-app-graphql-subscriptions-postgres/

How To Build A Real-Time App With GraphQL Subscriptions On Postgres

How To Build A Real-Time App With GraphQL Subscriptions On Postgres

Sandip Devarkonda

2018-12-10T14:00:23+01:00
2018-12-10T20:10:46+00:00

In this article, we’ll take a look at the challenges involved in building real-time applications and how emerging tooling is addressing them with elegant solutions that are easy to reason about. To do this, we’ll build a real-time polling app (like a Twitter poll with real-time overall stats) just by using Postgres, GraphQL, React and no backend code!

The primary focus will be on setting up the backend (deploying the ready-to-use tools, schema modeling), and aspects of frontend integration with GraphQL and less on UI/UX of the frontend (some knowledge of ReactJS will help). The tutorial section will take a paint-by-numbers approach, so we’ll just clone a GitHub repo for the schema modeling, and the UI and tweak it, instead of building the entire app from scratch.

All Things GraphQL

Do you know everything you need to know about GraphQL? If you have your doubts, Eric Baer has you covered with a detailed guide on its origins, its drawbacks and the basics of how to work with it. Read article →

Before you continue reading this article, I’d like to mention that a working knowledge of the following technologies (or substitutes) are beneficial:

ReactJS
This can be replaced with any frontend framework, Android or IOS by following the client library documentation.
Postgres
You can work with other databases but with different tools, the principles outlined in this post will still apply.

You can also adapt this tutorial context for other real-time apps very easily.

A demonstration of the features in the polling app that we’ll be building. (Large preview)

As illustrated by the accompanying GraphQL payload at the bottom, there are three major features that we need to implement:

Fetch the poll question and a list of options (top left).
Allow a user to vote for a given poll question (the “Vote” button).
Fetch results of the poll in real-time and display them in a bar graph (top right; we can gloss over the feature to fetch a list of currently online users as it’s an exact replica of this use case).

Front-end is messy and complicated these days. That’s why we publish articles, printed books and webinars with useful techniques to improve your work. Even better: Smashing Membership with a growing selection of front-end & UX goodies. So you get your work done, better and faster.

Explore Smashing Membership ↬

Challenges With Building Real-Time Apps

Building real-time apps (especially as a frontend developer or someone who’s recently made a transition to becoming a fullstack developer), is a hard engineering problem to solve.

This is generally how contemporary real-time apps work (in the context of our example app):

The frontend updates a database with some information; A user’s vote is sent to the backend, i.e. poll/option and user information (user_id, option_id).
The first update triggers another service that aggregates the poll data to render an output that is relayed back to the app in real-time (every time a new vote is cast by anyone; if this done efficiently, only the updated poll’s data is processed and only those clients that have subscribed to this poll are updated):

Vote data is first processed by an register_vote service (assume that some validation happens here) that triggers a poll_results service.
Real-time aggregated poll data is relayed by the poll_results service to the frontend for displaying overall statistics.

A poll app designed traditionally

This model is derived from a traditional API-building approach, and consequently has similar problems:

Any of the sequential steps could go wrong, leaving the UX hanging and affecting other independent operations.
Requires a lot of effort on the API layer as it’s a single point of contact for the frontend app, that interacts with multiple services. It also needs to implement a websockets-based real-time API — there is no universal standard for this and therefore sees limited support for automation in tools.
The frontend app is required to add the necessary plumbing to consume the real-time API and may also have to solve the data consistency problem typically seen in real-time apps (less important in our chosen example, but critical in ordering messages in a real-time chat app).
Many implementations resort to using additional non-relational databases on the server-side (Firebase, etc.) for easy real-time API support.

Let’s take a look at how GraphQL and associated tooling address these challenges.

What Is GraphQL?

GraphQL is a specification for a query language for APIs, and a server-side runtime for executing queries. This specification was developed by Facebook to accelerate app development and provide a standardized, database-agnostic data access format. Any specification-compliant GraphQL server must support the following:

Queries for reads
A request type for requesting nested data from a data source (which can be either one or a combination of a database, a REST API or another GraphQL schema/server).
Mutations for writes
A request type for writing/relaying data into the aforementioned data sources.
Subscriptions for live-queries
A request type for clients to subscribe to real-time updates.

GraphQL also uses a typed schema. The ecosystem has plenty of tools that help you identify errors at dev/compile time which results in fewer runtime bugs.

Here’s why GraphQL is great for real-time apps:

Live-queries (subscriptions) are an implicit part of the GraphQL specification. Any GraphQL system has to have native real-time API capabilities.
A standard spec for real-time queries has consolidated community efforts around client-side tooling, resulting in a very intuitive way of integrating with GraphQL APIs.

GraphQL and a combination of open-source tooling for database events and serverless/cloud functions offer a great substrate for building cloud-native applications with asynchronous business logic and real-time features that are easy to build and manage. This new paradigm also results in great user and developer experience.

In the rest of this article, I will use open-source tools to build an app based on this architecture diagram:

A poll app designed with GraphQL

Building A Real-Time Poll/Voting App

With that introduction to GraphQL, let’s get back to building the polling app as described in the first section.

The three features (or stories highlighted) have been chosen to demonstrate the different GraphQL requests types that our app will make:

Query
Fetch the poll question and its options.
Mutation
Let a user cast a vote.
Subscription
Display a real-time dashboard for poll results.

GraphQL request types in the poll app (Large preview)

Prerequisites

A Heroku account (use the free tier, no credit card required)
To deploy a GraphQL backend (see next point below) and a Postgres instance.
Hasura GraphQL Engine (free, open-source)
A ready-to-use GraphQL server on Postgres.
Apollo Client (free, open-source SDK)
For easily integrating clients apps with a GraphQL server.
npm (free, open-source package manager)
To run our React app.

Deploying The Database And A GraphQL Backend

We will deploy an instance each of Postgres and GraphQL Engine on Heroku’s free tier. We can use a nifty Heroku button to do this with a single click.

Heroku button

Note: You can also follow this link or search for documentation Hasura GraphQL deployment for Heroku (or other platforms).

Deploying Postgres and GraphQL Engine to Heroku’s free tier (Large preview)

You will not need any additional configuration, and you can just click on the “Deploy app” button. Once the deployment is complete, make a note of the app URL:

<app-name>.herokuapp.com

For example, in the screenshot above, it would be:

hge-realtime-app-tutorial.herokuapp.com

What we’ve done so far is deploy an instance of Postgres (as an add-on in Heroku parlance) and an instance of GraphQL Engine that is configured to use this Postgres instance. As a result of doing so, we now have a ready-to-use GraphQL API but, since we don’t have any tables or data in our database, this is not useful yet. So, let’s address this immediately.

Modeling the database schema

The following schema diagram captures a simple relational database schema for our poll app:

Schema design for the poll app. (Large preview)

As you can see, the schema is a simple, normalized one that leverages foreign-key constraints. It is these constraints that are interpreted by the GraphQL Engine as 1:1 or 1:many relationships (e.g. poll:options is a 1: many relationship since each poll will have more than 1 option that are linked by the foreign key constraint between the id column of the poll table and the poll_id column in the option table). Related data can be modelled as a graph and can thus power a GraphQL API. This is precisely what the GraphQL Engine does.

Based on the above, we’ll have to create the following tables and constraints to model our schema:

Poll
A table to capture the poll question.
Option
Options for each poll.
Vote
To record a user’s vote.
Foreign-key constraint between the following fields (table : column):

option : poll_id → poll : id
vote : poll_id → poll : id
vote : created_by_user_id → user : id

Now that we have our schema design, let’s implement it in our Postgres database. To instantly bring this schema up, here’s what we’ll do:

Download the GraphQL Engine CLI.
Clone this repo:
$ git clone clone https://github.com/hasura/graphql-engine

$ cd graphql-engine/community/examples/realtime-poll

Go to hasura/ and edit config.yaml:
endpoint: https://<app-name>.herokuapp.com

Apply the migrations using the CLI, from inside the project directory (that you just downloaded by cloning):
$ hasura migrate apply

That’s it for the backend. You can now open the GraphQL Engine console and check that all the tables are present (the console is available at https://<app-name>.herokuapp.com/console).

Note: You could also have used the console to implement the schema by creating individual tables and then adding constraints using a UI. Using the built-in support for migrations in GraphQL Engine is just a convenient option that was available because our sample repo has migrations for bringing up the required tables and configuring relationships/constraints (this is also highly recommended regardless of whether you are building a hobby project or a production-ready app).

Integrating The Frontend React App With The GraphQL Backend

The frontend in this tutorial is a simple app that shows poll question, the option to vote and the aggregated poll results in one place. As I mentioned earlier, we’ll first focus on running this app so you get the instant gratification of using our recently deployed GraphQL API , see how the GraphQL concepts we looked at earlier in this article power the different use-cases of such an app, and then explore how the GraphQL integration works under the hood.

NOTE: If you are new to ReactJS, you may want to check out some of these articles. We won’t be getting into the details of the React part of the app, and instead, will focus more on the GraphQL aspects of the app. You can refer to the source code in the repo for any details of how the React app has been built.

Configuring The Frontend App

In the repo cloned in the previous section, edit HASURA_GRAPHQL_ENGINE_HOSTNAME in the src/apollo.js file (inside the /community/examples/realtime-poll folder) and set it to the Heroku app URL from above:
export const HASURA_GRAPHQL_ENGINE_HOSTNAME = ‘random-string-123.herokuapp.com’;

Go to the root of the repository/app-folder (/realtime-poll/) and use npm to install the prequisite modules and then run the app:
$ npm install

$ npm start

Screenshot of the live poll app (Large preview)

You should be able to play around with the app now. Go ahead and vote as many times as you want, you’ll notice the results changing in real time. In fact, if you set up another instance of this UI and point it to the same backend, you’ll be able to see results aggregated across all the instances.

So, how does this app use GraphQL? Read on.

Behind The Scenes: GraphQL

In this section, we’ll explore the GraphQL features powering the app, followed by a demonstration of the ease of integration in the next one.

The Poll Component And The Aggregated Results Graph

The poll component on the top left that fetches a poll with all of its options and captures a user’s vote in the database. Both of these operations are done using the GraphQL API. For fetching a poll’s details, we make a query (remember this from the GraphQL introduction?):

query {
poll {
id
question
options {
id
text
}
}
}

Using the Mutation component from react-apollo, we can wire up the mutation to a HTML form such that the mutation is executed using variables optionId and userId when the form is submitted:

mutation vote($optionId: uuid!, $userId: uuid!) {
insert_vote(objects: [{option_id: $optionId, created_by_user_id: $userId}]) {
returning {
id
}
}
}

To show the poll results, we need to derive the count of votes per option from the data in vote table. We can create a Postgres View and track it using GraphQL Engine to make this derived data available over GraphQL.

CREATE VIEW poll_results AS
SELECT poll.id AS poll_id, o.option_id, count(*) AS votes
FROM (( SELECT vote.option_id, option.poll_id, option.text
FROM ( vote
LEFT JOIN
public.option ON ((option.id = vote.option_id)))) o

LEFT JOIN poll ON ((poll.id = o.poll_id)))
GROUP BY poll.question, o.option_id, poll.id;

The poll_results view joins data from vote and poll tables to provide an aggregate count of number of votes per each option.

Using GraphQL Subscriptions over this view, react-google-charts and the subscription component from react-apollo, we can wire up a reactive chart which updates in realtime when a new vote happens from any client.

subscription getResult($pollId: uuid!) {
poll_results(where: {poll_id: {_eq: $pollId}}) {
option {
id
text
}
votes
}
}

GraphQL API Integration

As I mentioned earlier, I used Apollo Client, an open-source SDK to integrate a ReactJS app with the GraphQL backend. Apollo Client is analogous to any HTTP client library like requests for python, the standard http module for JavaScript, and so on. It encapsulates the details of making an HTTP request (in this case POST requests). It uses the configuration (specified in src/apollo.js) to make query/mutation/subscription requests (specified in src/GraphQL.jsx with the option to use variables that can be dynamically substituted in the JavaScript code of your REACT app) to a GraphQL endpoint. It also leverages the typed schema behind the GraphQL endpoint to provide compile/dev time validation for the aforementioned requests. Let’s see just how easy it is for a client app to make a live-query (subscription) request to the GraphQL API.

Configuring The SDK

The Apollo Client SDK needs to be pointed at a GraphQL server, so it can automatically handle the boilerplate code typically needed for such an integration. So, this is exactly what we did when we modified src/apollo.js when setting up the frontend app.

Making A GraphQL Subscription Request (Live-Query)

Define the subscription we looked at in the previous section in the src/GraphQL.jsx file:

const SUBSCRIPTION_RESULT = `
subscription getResult($pollId: uuid!) {
poll_results (
order_by: option_id_desc,
where: { poll_id: {_eq: $pollId} }
) {
option_id
option { id text }
votes
}
}`;

We’ll use this definition to wire up our React component:

export const Result = (pollId) => (
<Subscription subscription={gql`${SUBSCRIPTION_RESULT}`} variables={pollId}>
{({ loading, error, data }) => {
if (loading) return

Loading…</p>;
if (error) return

Error :</p>;
return (
<div>
<div>
{renderChart(data)}
</div>
</div>
);
}}
</Subscription>
)

One thing to note here is that the above subscription could also have been a query. Merely replacing one keyword for another gives us a “live-query”, and that’s all it takes for the Apollo Client SDK to hook this real-time API with your app. Every time there’s a new dataset from our live-query, the SDK triggers a re-render of our chart with this updated data (using the renderChart(data) call). That’s it. It really is that simple!

Final Thoughts

In three simple steps (creating a GraphQL backend, modeling the app schema, and integrating the frontend with the GraphQL API), you can quickly wire-up a fully-functional real-time app, without getting mired in unnecessary details such as setting up a websocket connection. That right there is the power of community tooling backing an abstraction like GraphQL.

If you’ve found this interesting and want to explore GraphQL further for your next side project or production app, here are some factors you may want to use for building your GraphQL toolchain:

Performance & Scalability
GraphQL is meant to be consumed directly by frontend apps (it’s no better than an ORM in the backend; real productivity benefits come from doing this). So your tooling needs to be smart about efficiently using database connections and should be able scale effortlessly.
Security
It follows from above that a mature role-based access-control system is needed to authorize access to data.
Automation
If you are new to the GraphQL ecosystem, handwriting a GraphQL schema and implementing a GraphQL server may seem like daunting tasks. Maximize the automation from your tooling so you can focus on the important stuff like building user-centric frontend features.
Architecture
As trivial as the above efforts seem like, a production-grade app’s backend architecture may involve advanced GraphQL concepts like schema-stitching, etc. Moreover, the ability to easily generate/consume real-time APIs opens up the possibility of building asynchronous, reactive apps that are resilient and inherently scalable. Therefore, it’s critical to evaluate how GraphQL tooling can streamline your architecture.

Related Resources

You can check out a live version of the app over here.
The complete source code is available on GitHub.
If you’d like to explore the database schema and run test GraphQL queries, you can do so over here.

(rb, ra, yk, il)