Sangre

Sangre streams your backend queries in realtime to your clients minimizing the load via diffs.

Sangre lives with your current backend framework (expressjs, django, rails, etc.)

About The Project • How it works • Installation • Contact • Acknowledgments

Example

This example is based on dart but feel free to imagine in your own language.

void main() async {
  final app = await setupMyApp(); // expressjs, django, rails

  // Setup node
  final followedNode = await DBNode('users').get('id', 1).joinMany(
        'followeds',
        fromNode: DBNode('users').joinMany(
          'places',
          fromNode: DBNode('places'),
        ),
      );

  // Plug the node to an endpoint
  app.sangre('/followeds', followedNode);

  await app.listen();
}

This exposes a websocket endpoint streaming the user (id==1) with its followed users populated with their places.

The websocket streams the updates as this data changes in the database. See working example here.

About The Project

Sangre provides a generic solution for streaming complex backend queries to clients in realtime.

A complex query is an arbitrary nested query of structured database data and processing of this data in your native backend language (js, python, etc.)

The result of such query is streamed to client using incremental updates, minimizing network load, and enabling offline sync.

Typical use case is a client-server topology, a mobile or web app consuming an API (expressjs, django, rails, etc.) in front of a database (postgres, mongo, mysql, etc.). You want realtime data in your app without all the complexity of developing your own data sync.

Sangre implements all the following features and let the developers focus of business logic.

How it works

Sangre is an acyclic graph of operator nodes acting on data. This data flows in those nodes as streams of data for reactivness.

Nodes may be filters, joins, populators. Root nodes provide data from the underling database (typically postgres) and listen to changes (via supabase realtime) in order to spread them down the data flow. Leaf nodes are the endpoints consumed by client apps (via websocket).

Limitations (alpha)

Note: Dart is temporary, TypeScript might be the final implementation. I just happened to have my head in Dart when starting the PoC.

At this point, Sangre is just a PoC. A lot of shortcuts have been taken to produce a working example. Here are known limitations, if you think about any other, please reach me out via my contact info.

Diff algorithm is currently JSON patch. This can be easily changed for a more readable or effecient one (myers, histogram, yours ?)

Installation (TODO)

Note : only postgres supported ATM (more to come)

Note : You can use docker-compose.yml to get a working example running

1. Enable postgres replication (TODO)

ALTER SYSTEM SET wal_level = logical;
CREATE PUBLICATION supabase_realtime FOR ALL TABLES;

2. Install realtime broker (TODO)

insert supabase realtime installation steps

Contact

Project Link: https://github.com/pomarec/sangre

Acknowledgments

• Inspirations :

  • NoriaDB : Huge thanks to Jon Gjengset for clearing up my mind about this topic (Whitepaper). Sangre is not an implementation of this paper though.
  • All the work done around materialized views and dataflows (Raw list of sources)

• Dart librairies :

  • alfred
  • dartz
  • json_patch
  • postgres
  • realtime_client
  • rxdart
  • web_socket_channel
  • flutter
  • flex_color_scheme
  • hovering

• Blood icons created by Freepik – Flaticon

GitHub

View Github