Skip to main content

Architecture

Architecture Overview​

RAW connects to data sources - databases, data lakes with files, or APIs - and provides a SaaS platform to query, integrate, deliver and share data.

RAW System Context

Users can be:

  • Administrators, which manage the platform;
  • Developers, also known as Producers, which produce the APIs;
  • Consumers, which can be individuals or machines that consume the APIs built by the producers.

RAW System Overview

Components​

Snapi - A declarative language for querying and manipulating data. Snapi supports non-relational, complex and nested data structures, i.e. JSON, XML. With Snapi you can query APIs and Log files together with databases in a single query without having to land data and transform it first. For more information see our Snapi Language Reference Guide.

YAML API Configuration - RAW uses a simple YAML file mechanism to configure API endpoints to execute Snapi queries. Security, Resource Groups and Metadata are defined. For more information see our YAML Reference Guide

VS Code Development Environment - RAW uses the popular Microsoft Visual Studio Code, as an IDE to enable Snapi query code to be developed, APIs to be configured and executed from within a single place. For more information see our RAW VS Code Extension Reference Guide

RAW DataOps Environment - RAW harnesses GitHub for code management and DevOps using GitHub Actions for CI/CD. RAW syncs with your GitHub Repository automatically so that APIs are delivered immediately on a Git Push, including any branches for testing purposes For more information see our GitHub Integration Reference Guide

RAW Execution Engine - A cloud-based query engine designed for scale-out execution for distributed, heterogenous data and complex data types. Based on unique monoid comprehension calculus, the engine is built with state-of-the-art 'JIT' code generation techniques, creates high-performance code employing data and code caches and associated algorithms.

Resource Groups - RAW Execution Engines are available in multiple sizes using cloud infrastructure to ensure that the needs of the Snapi query can be matched better with the class of service. Resource Groups include appropriate CPU, Memory and Storage and are categorized as Standard(S), Large(L) and Extra-Large(XL)

RAW Admin UI - A web-based administration portal for Producers. Here you can monitor your Repositories/APIs, administer users and share data access, as well as view logs for access, resource usage, Git sync, etc. For more information see our Admin Console Reference Guide

RAW Catalog UI - A web portal displaying the API Catalog for authenticated Consumers to browse. For more information see our Catalog Console Reference Guide