Aleksander Reitsakas
Technical Reviewers: Tony Shannon, Saša Kovačević, Riham Moawad, Riham Fahmi, Aare Laponin, Manish Srivastava, Palab Saha, Surendra Singh Sucharia, Arvind Gupta, Gayatri. P., Shivank Singh Chauhan, Gavin Lyons
| External links updated | | 1.0.0 RC1 |Aleksander Reitsakas, Taylor Downs, Dr. P. S. Ramkumar, Aleksa Krolls
Reviewers:
Neil Roy, Aare Lapõnin, Fergal Marrinan, Surendrasingh Sucharia, Amy Darling
1.0.0
previously 23Q4
Software modules that can be deployed and combined in a standardized manner. Each Building Block is capable of working independently, but they can be combined to do much more. Building Blocks are composable, interoperable software modules that can be used across a variety of use cases. They are standards-based, preferably open-source, and designed for scale. Each Building Block exposes a set of services in the form of REST APIs that can be consumed by other Building Blocks or applications.
Ingress access is access from external applications to GovStack Building Blocks and applications.
Egress access is access from within GovStack Building Blocks and applications to external applications.
| | **Use Case** |A use case is a piece of functionality described as a sequence of actions (steps) to achieve a specific goal in a specific context of usage. E.g., in one use case, the Information Mediator Building Block may be used to let a Building Block access a service provided by another, or in a different use case may be used to relay an event notification from one Building Block to several other Building Blocks via a Publisher-Subscriber (Pub/Sub) model.
Each use case may involve a collection of modules or Building Blocks. A relatively small set of these Building Blocks can be readily applied to a wide variety of use cases in low-resource settings.
| | **API** | An application programming interface ([API](https://en.wikipedia.org/wiki/API)) is a connection between computers or between computer programs. It is a type of software interface, that offers a service to other pieces of software. A document or standard that describes how to build such a connection or interface is called an API specification. A computer system that meets this standard is said to implement or expose an API. The term API may refer either to the specification or to the implementation. | ## 3.5 Service Access | Term | Description | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Member** | A member is an organization that is authorized to communicate via the Information Mediator for a particular GovStack implementation. | | **Application** | An application is a running instance containing one or more Building Block instances and zero or more use case implementations. An application uses an Information Mediator Building Block to communicate with other Building Blocks or applications. An application typically has a single responsibility. | | **Service** | A service is a minimal piece of functionality provided by a Building Block or use case implementation. A service can be local (inside one application) or remote (between different applications). Remote services are consumed using REST protocol and they are described by OpenAPI specification. For example, the “registration” service might be accessed at a particular URL and allow a requester to “register a patient” by sending a POST request with proper patient data. | | **Security Server** | A security server is the main piece of software that is responsible for implementing the “service access” layer of the Information Mediator. This software acts as a gateway and is responsible for mediating requests between various members, applications, and services. It might be not just a single piece, but also a clustered or serverless deployment. | ## 3.6 Pub/Sub | Term | Description | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Publisher** | A Publisher produces events and sends them to rooms. Each event has an event type associated with it. Publishers can produce events of different types. | | **Room** | A Room is a Pub/Sub entity that handles the distribution of events. Each Room has a set of connected event types (E.g., the “birth” Room might contain three event types: “new\_birth”, “birth\_complication”, and “infant\_death”). | | **Subscriber** | A Subscriber can process events of a certain event type. Subscribers are independent of each other and their business logic is different (as rule). Each subscriber processes events from their own perspective. | # 4 Key Digital Functionalities Key Digital Functionalities describe the core (required) functions that this Building Block must be able to perform. The Information Mediator Building Block should provide the following functionalities, organized across 6 different components or layers ## 4.1 Service Access Service access is the facility through which different GovStack Building Blocks and applications may communicate securely with each other. The service access layer should provide an administrative interface that can be used to define users and roles and what access each should have. This layer should also include a registration interface which allows new members to onboard to the Information Mediator. The Service Access layer will also provide functionality to allow or restrict secure access to various resources, route requests to the correct providers, connect service requestors to providers, and enforce access privileges and security policies ## 4.2 Directory Services The Directory service allows applications to discover resources or functionalities that are made available by the system. An application may query the Directory service to discover organizations, services, and APIs that are available to them ## 4.3 Pub/Sub Service The Pub/Sub service is a facility through which applications may publish and subscribe to different events identified by unique message types, enabling more efficient and resilient communication and application design. It should provide broadcast/multicast capabilities to facilitate faster, more resilient application design. ## 4.4 Logging Services The Information Mediator will manage a message log which will provide comprehensive audit trails for all services requests and access. ## 4.5 Monitoring Services The Monitoring services will provide both operational and environmental reporting and dashboards, allowing users to discover the volume of requests to particular services or organizations as well as environmental reports on hardware/network usage. ## 4.6 Scaling/Throughput Services The Information Mediator should support scaling services such as load balancers and allow for the addition of additional servers to manage increased system use. ## Out-of-scope requirements Note that the Information Mediator is not responsible for manipulating the payloads sent to and from various applications—in a sense, it is both the postal service and the roads/bridges/train tracks—but it does not read the contents of your mail. The Information Mediator Building Block does not handle communication between human users and applications. The Information Mediator Building Block does not handle ingress and egress access from external applications to and from GovStack’s internal components (This may be handled by a public API Gateway in the domain of the Security Building Block). The current specification does not assume the possibility of different Information Mediator implementations working together. In one instance of the GovStack ecosystem, only one implementation of the Information Mediator Building Block can exist. The following requirements have been identified as “out of scope” because they fall into the realm of the Workflow Building Block or the Security Building Block. * Supports the handling of transactions and associated communication errors and exceptions. * Translates data from one format to another, and interoperates with handshake protocols to enable interoperability between different ICT (Information and communication technology) Building Blocks duplications. * Map data structures and fields from the identification system to the registration system and vice versa. * Hold authentication and credentials for each system. * Allow the definition of steps for a particular transaction. * Provide an API for both systems to access – and execute all necessary steps for that transaction (including error handling, retries, and notifications). * Provide an API for external systems to access GovStack Building Blocks. ## Future Scope In future iterations of this specification, we may take into consideration more broad API-management standards which include multiple domains, such as those proposed by the United Kingdom's Government in their [Defining an API management strategy](https://www.gov.uk/guidance/defining-an-api-management-strategy) and the section on [Gov.UK’s API Management Strategy Document](#4.3.3.2-gov.uks-api-management-strategy-document). # 5 Cross-Cutting Requirements This section will highlight important requirements or describe any additional cross-cutting requirements that apply to this Building Block. ## 5.1 Requirements The cross-cutting requirements described in this section are an extension of the cross-cutting requirements defined in the [Architecture Blueprint document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements) and the [Security Requirements](https://govstack.gitbook.io/specification/v/1.0/security-requirements). This section will describe any additional cross-cutting requirements for this Building Block or differences with the Architecture Blueprint. ### 5.1.1 GET and PUT APIs must be Idempotent (REQUIRED) Application APIs will contain POST endpoints which are not idempotent. GET/PUT/DELETE APIs are idempotent relative to Information Mediator Building Block, but the idempotents of intermediated services depend on the service provider and cannot be specified at this level. ### 5.1.2 Databases should not Include Business Logic (RECOMMENDED) Databases should not include business logic. We propose that this is a design recommendation which intends to make business logic all live in a clearly visible and accessible location. This may not always be followed, as the benefits of stored procedures (e.g., in reducing database transaction round-trips, etc.) may sometimes outweigh this general design recommendation. ### 5.1.3 Design for Asynchronous First (RECOMMENDED) The Service Access Layer of the Information Mediator is synchronous first. The Pub/Sub Layer is asynchronous. ### 5.1.4 Use Standardized Data Formats for Interchange (REQUIRED) Standard formats are used for communication with other Building Blocks. Inside Building Block non-standard protocols can be used. ### 5.1.5 Use Web Hooks for Callbacks (REQUIRED) OpenAPI spec 3.0 is supported in the first version. Moving to OpenAPI 3.1 is planned for the future. ## 5.2 Performance Requirements The Performance Requirements specify the basic parameters that an implementing government might use to establish performance requirements for scalability, throughput, and response times when reasonable/necessary. For example: * Minimum Throughput = 100 requests/sec. * Maximum Latency = 1 sec. * Concurrency = 1000 concurrent requests. * All solutions MUST be able to monitor and report, including but not limited to, resource consumption, throughput, latency, average latency, queue depth/backlogs, etc. * All of these indicators MUST be available through an administrative API. * Ideally, all Building Blocks should be able to run a “monitoring agent” which handles reporting out logs, requests, Building Block-specific indicators, etc. to a monitoring service (e.g. [Datadog](https://www.datadoghq.com/)) * The local monitoring agent should be configurable via web interface. * Retries and back-off strategies must be configurable. * Specific “[Scaling/Throughput](https://mediator.govstack.global/development-16/pages/LPb9h8GPp7kffWEcTJ2t#6.6-scaling-throughput)” requirements are in the functional requirements section. ## 5.3 Standards The following standards are applicable to data structures in the Information Mediator Building Block: # 6 Functional Requirements This section lists the technical capabilities of this Building Block. The functional requirements section lists the technical capabilities that this Building Block should have. These requirements should be sufficient to deliver all functionality that is listed in the [Key Digital Functionalities](/development-16/4-key-digital-functionalities) section. The Information Mediator is used only for interaction between Applications/Building Blocks for the transfer of data at an API level only. Details in the [Architecture and Nonfunctional Requirements document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements/6-onboarding). These functional requirements do not define specific APIs, they provide a list of information about functionality that must be implemented within the Building Block. Detailed design and feature lists of these blocks can be customized by developers to optimally match specific target implementation needs. ## 6.1 Administrative Interface * There should be different types of administrative roles that provide different levels of access (RECOMMENDED) * There should be administrative roles on different levels of the ecosystem (RECOMMENDED): * global administrator role of the whole ecosystem, * local administrator role of Member, * an administrative role for configuring an Application. * There should be an ecosystem-level administrative interface that allows an Information Mediator administrator (with appropriate authentication) to register/deregister members and applications (RECOMMENDED) * There should be a security-server-level interface that allows an administrator for a single security server to manage members, applications, and services that live under a single security server (RECOMMENDED) * Open IAM (or another Identity and Access Management Solution) must be able to create/edit/delete admin users for the Information Mediator interface. [Security Building Block specification 1.0.1 section 6-1](https://govstack.gitbook.io/specification/v/1.0/security-requirements/6-security-building-block-modules) (REQUIRED) ## 6.2 Registration Services * Provide a mechanism for registering a member/organization, where the new member provides details for registration and the administrator verifies and accepts the request for registration (REQUIRED) * Provide a mechanism for registering an application where the new member provides details for registration and the administrator verifies and accepts the request for registration (REQUIRED) * Provide a mechanism for registering a service (API endpoints), which includes an OpenAPI 3.0 specification document and description, along with the ability to enable and disable endpoints (REQUIRED) * Provide a mechanism for application developers to access the directory service (described below) to select the services that they want to consume (RECOMMENDED) * The request/approval/addition of allowed consumers process is a business-first process with manual steps (The hard part is negotiating data sharing agreements and signing contracts when that is completed the Information Mediator administrator can easily modify the allowed consumers' list manually) * An application must specify which member/application/service they want to access * The provider of that service must decide if the consumer is allowed to access the service. * Once approved, the requesting application will be added to the list of allowed applications for the requested service ## 6.3 Accessing Services * The Information Mediator must allow a service to make a request to another service using REST calls (REQUIRED) * The REST request must use HTTPS/TLS to the local Information Mediator security server with headers that identifies itself at the application level. * The components of the request must be (with color code): * Security server URL; * API version; * Instance (e.g., Country); * Domain of member; * Member (e.g., Ministry of X); * Application; * Service (OpenAPI file); * Path * Endpoint. * Query parameters. **Example Only**\ *Sample GET Request:*\ url-of-local-information-mediator-security-server/r1/INDIA/GOV/ministry-of-agriculture-karnataka/market-linkages-app/inventory-service/v1/check-level/apples?fresh=true\ *Response*: { data: 7 }\ \ *Sample POST Request:*\ url-of-local-information-mediator-security-server/r1/INDIA/PVT/tata-buyers-corp-karnataka/small-farmer-buyers-app/inventory-service/v1/supply/apples\ *With body*: {“qtyAvailable”: 4}\ *Response*: { result: “Stock level report created.” }\ \ *How to interpret the above request paths:*\ SECURITY-SERVER-URL/r1/INSTANCE/DOMAIN/MEMBER/APPLICATION/SERVICE/PATH * Note that all applications are making requests to the security server, which runs over the private network segment, rather than making requests to other applications directly over the public internet. This is the desired situation and is one of the main principles of the Information Mediator Building Block architecture and the architecture of the Security Server ## 6.4 Directory Services * At development time, to see which resources are available on GovStack, the administrator/developer of application A may send requests to the security server to see organizations, services, and OpenAPI specifications. (RECOMMENDED) * A view layer allowing for easy exploration of ALL clients, applications, and services should be provided. (Note that, “under the hood”, this layer may make use of the APIs described above or be implemented via a separate API). (RECOMMENDED) ## 6.5 Pub/Sub Service * The Pub/Sub layer must provide a mechanism for registering Rooms (RECOMMENDED) * The Pub/Sub layer must provide a mechanism for registering event types (RECOMMENDED) * For each event type, an OpenAPI description of a service with JSON schema for the required payload shape must be defined to create an event of the type (RECOMMENDED) * An interface for registering and viewing event types must be provided (RECOMMENDED) * An application must be able to publish a message by making a POST request with a valid JSON payload as the body and specify the message type to a Room provided by the Pub/Sub layer service (RECOMMENDED) * Published messages should be delivered to all subscribers (RECOMMENDED) * Published messages should be stored so that delivery may be done and retried asynchronously, e.g. if certain Subscribers are offline (RECOMMENDED) * Subscribers should be able to configure their retry strategies, overriding the default exponential backoff on retriable errors ([Google Pub/Sub example](https://cloud.google.com/pubsub/docs/handling-failures)) * If an active subscription exists but an error is received when forwarding a message to that service, the Pub/Sub layer should retry N times with a standard backoff. (An exponential backoff may be the default approach.) (RECOMMENDED) * The backoff and retry strategy should be configurable by an administrator with access to the Pub/Sub layer application * If a message cannot be delivered the Pub/Sub layer should drop that message (RECOMMENDED) * The Pub/Sub layer should provide an admin user interface to help create/manage subscriptions with the data below (RECOMMENDED) * The Pub/Sub layer should allow an administrator to view a list of active subscriptions (RECOMMENDED) * For registration via API, an application must be able to make a POST request to a service exposed by the Pub/Sub layer which defines which endpoints certain event types should be sent to (RECOMMENDED) * All events received and delivered must have a unique ID (RECOMMENDED) * All events received by the Pub/Sub layer must be logged or added to a log sync and those log entries must contain event metadata including the sender, timestamp, and event type, but may not include the event payload (RECOMMENDED) * All event delivery attempts must be logged or added to a log sync (RECOMMENDED) * For every event message received, the Information Mediator sends back an acknowledgment with the ID of the event to the respective Publisher (RECOMMENDED) * There must be a possibility to search and view the status of messages, for example, a message with type X to seven subscribers had been successfully delivered to all seven (RECOMMENDED) ## 6.6 Logging Services * The Information Mediator Building Block maintains a message log (REQUIRED) * The purpose of the message log is to provide means to prove the reception of a regular request or response message to a third party. Messages exchanged between Information Mediator Building Block are signed and encrypted. For every regular request and response, the security server produces a completely signed, and timestamped document. At a minimum, the log must store metadata that identifies a specific message, the status of transaction carried out on that message by Information Mediator Building Block, along with source ID and date time stamp. * The Information Mediator Building Block must have full audit trail capabilities (REQUIRED) * The Information Mediator Building Block keeps an audit log. The audit log events are generated by the user interface when the administrator changes the system's state or configuration. The administrator's actions are logged regardless of whether the outcome was a success or a failure. The system must be capable of emitting statistical reports for a given organization, application, service, or consumer and status. ## 6.7 Monitoring Services * The Information mediator must provide operational monitoring including information about requests and service health, (REQUIRED) * Operational monitoring provides details about the requested exchange, such as the ID-s of the client and the service, various attributes of the message read from the message header, request and response timestamps, sizes, etc., but not the actual payload of messages. * The operational monitoring daemon collects and shares operational monitoring data of the Information Mediator Building Block as part of request exchange, shares this data, and calculates and shares health statistics such as the timestamps and the number of successful/unsuccessful requests, various metrics of the duration and message size of the requests, etc. * The operational monitoring daemon makes operational and health data available to the owner of the security server, regular clients, and the central monitoring client via the security server. For example, local health data may be made available for external monitoring systems. * The owner of the security server and the central monitoring client are able to query the records of all clients. For a regular client, only the records associated with that client are available. * The Information mediator must provide environmental monitoring services such as CPU load, disk space, and traffic (REQUIRED) * Environmental monitoring provides details of the security servers such as operating system, memory, disk space, CPU load, traffic load, running processes, installed packages, etc. in a chosen date range. * Environmental monitoring provides a standard endpoint that can be accessed with a client (e.g. Java's console application if using Java Management Extensions). * It is possible to limit what allowed non-owners can request via environmental monitoring data requests. The security server owner will always get the full data set as requested. ## 6.8 Scaling/Throughput Services * The Information Mediator Building Block should support provider-side high availability setup if needed (RECOMMENDED) * Busy production systems may need a scalable performance in addition to high availability. The Information Mediator Building Block supports external load balancing mechanisms to address both of these problems simultaneously. A load balancer is added in front of a security server cluster to route the requests based on a selected algorithm. (RECOMMENDED) * The team in charge of deploying the security server application on their hardware must consider the network infrastructure including a load balancer, etc. The requirements for network infrastructure are to be described in the Information Mediator Building Block implementation guide. (REQUIRED) # 7 Data Structures This section provides information on the core data structures/data models that are used by this Building Block. The resource model shows the relationship between data objects that are used by the Information Mediator Building Block. The data elements provide detail for the resource model. All data element schemas can be viewed, commented on, and modified in the [schemas section of the Information Mediator Building Block GitHub repository](https://github.com/GovStackWorkingGroup/bb-information-mediator/tree/1.0-QA/schemas). ## 7.1 Resource Model The Mediator Building Block key element is Service. The Service is used by a consuming Building Block or an application and offered by a provider Building Block or Application. Both Provider and Consumer must be Members of the Mediator Building Block. Members of the Information Mediator Building Block can be an organization (governmental or not, business or not) or a person (citizen as a rule).  To become a Member of Mediator Building Block participants must fulfill declared requirements and apply for onboarding. In the process of onboarding a Member is registered with Mediator Building Block and gets credentials to connect to Mediator Building Block. Normally it is done the way that a Member provides a certificate of recognized Certification Authority (CA), and requests signed with this certificate are considered legitimate requests of the Member. A member entity can access the Pub/Sub configuration and register a room to publish its own event type as a publisher through its own admin. Members can browse a directory of Services available in the Mediator Building Block. Each service is described in OpenAPI. ## 7.2 Data Elements ### 7.2.1 Member **Fields:** | Name | Type | Description | | ------------ | ----------- | -------------------------------------------------------- | | class | enumerator | namespace for member\_code, e.g. GOV or COM | | code | string | unique id of member in registry defined by member\_class | | signing\_key | string | name of key used to sign on behalf of Member | | certificate | certificate | certificate of member issued by trusted CA | Schema reference [member.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/main/schemas/member.json) ### 7.2.2 Application **Fields:** | Name | Type | Description | | ---------------- | ----------- | ------------------------------------------- | | code | string | unique name of application in member scope | | connection\_type | string | protocol used for connection: http or https | | certificate | certificate | TLS certificate | Schema reference [application.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/main/schemas/application.json) ### 7.2.3 Service **Fields:** | Name | Type | Description | | ---------------- | ------ | ---------------------------------------------- | | code | string | unique name of OpenAPI in scope of application | | description\_url | url | location of OpenAPI service description | | service\_url | url | service provisioning network address | | ACL | object | description of access rights | Schema reference [service.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/main/schemas/service.json) ## 7.3 PubSub Layer ### 7.3.1 Resource Model The Resource Model is an extension of the Access Layer model:  ### 7.3.2 Data Elements #### 7.3.2.1 Event An event is a message—a set of data sent to a topic. Each event has an id. The event corresponds to the message. Data elements of an event are described by event type OpenAPI description. #### 7.3.2.2 Event Type An event type is schema definition for an event. Each event type is owned by a Room of a certain authority. (E.g. the Ministry of Health might own Room with the “new\_birth” event type and define its schema.) #### 7.3.2.3 Publisher A candidate application playing the role of **IM-Publisher** must be able to emit events to a specific Room. | Data Element | Default format | Description | | ------------ | -------------- | -------------------------------- | | id | string | application id of this publisher | | name | string | OPTIONAL | #### 7.3.2.4 Room *(N.B., this is often called a "topic" and we may shift to that in later versions.)* A candidate application playing the role of **IM-Room** must handle the distribution of events. Each Room has a set of connected event types (e.g., the “birth” room might contain three event types: “new\_birth”, “birth\_complication”, and “infant\_death”). A room is located in the member’s local Information Mediator Building Block implementation and the member is responsible for all types of events in that particular room. | Data Element | Default format | Description | | ------------ | -------------- | --------------------------- | | id | string | application id of this room | | name | string | OPTIONAL | #### 7.3.2.5 Subscriber A candidate application playing the role of **IM-Subscriber** must be able to process events. It is done by defining a Service that will be called by Room to deliver an event. Subscribers are independent of each other and their business logic differs (as a rule). Each subscriber processes events from their own perspective. | Data Element | Default format | Description | | ------------ | -------------- | --------------------------------- | | id | string | application id of this subscriber | | name | string | OPTIONAL | #### 7.3.2.6 Subscription | Data Element | Default format | Description | | -------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | id | string | subscription id | | room\_id | string | room id | | subscriber\_id | string | subscriber id | | event\_type | string | A filter expression that allows a subscriber to subscribe to only certain message\_types that are published to the room they're subscribed to. | | mode | enum | delivery mode | | details | object | details of delivery, like time to live, repetition policy, etc. | # 8 Service APIs This section provides a reference for APIs that should be implemented by this Building Block. The [GovStack non-functional requirements document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements/6-onboarding) provides additional information on how 'adaptors' may be used to translate an existing API to the patterns described here. This section also provides guidance on how candidate products are tested and how GovStack validates a product's API against the API specifications defined here. The tests for the Information Mediator Building Block can be found in [this GitHub repository](https://github.com/GovStackWorkingGroup/bb-information-mediator/tree/main/test/openAPI). The majority of functions provided by the Information Mediator Building Block are either defined in the “service access flow” or configured by the administrator via the web User Interface. There is, however, a “Directory Service” which can provide listings of clients, methods, and available API specifications for services on the Information Mediator. The directory is managed by admins of members. The directory service centralizes and offers knowledge of all enrolled members and their services along with the information necessary to bind a third-party application as a consumer of that service. These services are described here: * [API metadata](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/main/api/GovStack_IM_Directory_Services_API.yaml) and changes to the API definitions can be made by submitting a pull request on this repository. Additional APIs may be implemented by the Building Block, but the listed APIs define a minimal set of functionality that should be provided by any implementation of this Building Block. The services can be accessed via the following Service APIs: ## 8.1 Service Access The full API definition of all available services is the set of all available OpenAPI descriptions. One can take any of the available OpenAPI descriptions and call service according to that description. This call must be forwarded to IM local Security Server and path part of the called URL must begin with the address of service in the form `/r1/{instanceId}/{memberClass}/{member}/{application}/{service}/` followed by the service path with possible query parameters. The address of the service may be already listed in the OpenAPI description or must be added to the path if not provided by OpenAPI. ## 8.2 Directory Services ### 8.2.1 Member Discovery At development time, to see which organizations are available on GovStack, an administrator of application A sends a GET request to the security server: url-of-local-information-mediator-security-server/r1/listClients The response is an **array of organizations** with descriptions. API MAY implement paging of output. ## List clients defined in the instance > Clients of the security server have the capability to obtain a list\ > of potential service providers within a GovStack instance, including\ > both members and applications.\ > To do so, they should initiate an HTTP GET request to the security\ > server.\ > The specific request URL will be either \`This API takes provided filters from query parameters and returns a subset of access policies configured on the IM building block, based on the filters.\
\The result is paged, with page size based on the number of service and allowed application pairs. For example, when pageSize = 10 and the IM BB defines three services: A, B and C, each of which has 8 clients, the result will be split over 3 pages:\
\
Page 1: Service A, with all of its 8 allowed applications, Service B with 2 allowed applications.\
Page 2: Service B, with 6 allowed applications left over, Service C with 4 applications.\
Page 3: Service C, with 4 applications.\
\
Implementations can define additional query parameters for result display (such as ordering) and how next page tokens are implemented. It is expected that the next page token is not readable by API users and will be provided verbatim when querying the next page of results. \
```json {"openapi":"3.0.0","info":{"title":"GovStack IM Configuration Management API","version":"1.0.1"},"tags":[{"name":"rights","description":"Access rights management"}],"security":[{"auth":["read:rights"]}],"components":{"securitySchemes":{"auth":{"type":"oauth2","flows":{"authorizationCode":{"authorizationUrl":"https://example.com/oauth/authorize","tokenUrl":"https://example.com/oauth/token","scopes":{"read:conf":"Grants read access to IM configuration","create:conf":"Grants permission to create IM configuration","write:conf":"Grants write access to IM configuration","read:rights":"Grants permission to see what services are allowed","write:rights":"Grants permission to allow/deny IM services"}}}}},"schemas":{"PaginatedResult":{"type":"object","properties":{"nextPageToken":{"type":"string"}}},"ServiceAccessRights":{"type":"array","items":{"type":"object","required":["memberClass","memberCode","applicationId","serviceId","rights"],"properties":{"memberClass":{"description":"Kind of organisation. Namespace for organisation {code}","type":"string"},"memberCode":{"description":"Registration number (or identifier) of the organisation in {memberClass} namespace","type":"string"},"applicationId":{"type":"string","description":"Name of application"},"serviceId":{"description":"Name of service/API","type":"string"},"rights":{"description":"Applications that are allowed or denied access","type":"array","items":{"$ref":"#/components/schemas/AllowedApplication"}}}}},"AllowedApplication":{"description":"An identified member application that has the permission to access a service.","type":"object","required":["member","application"],"properties":{"memberClass":{"description":"Member or member group identifier to whom access is granted","type":"string"},"memberCode":{"description":"Member or member group identifier to whom access is granted","type":"string"},"applicationId":{"description":"Application of the member that to whom access is granted","type":"string"}}}}},"paths":{"/api/v1/rights/allow":{"get":{"tags":["rights"],"summary":"Retrieve configured access permissions (allow list)","description":"This API takes provided filters from query parameters and returns a subset of access policies configured on the IM building block, based on the filters.
The result is paged, with page size based on the number of service and allowed application pairs. For example, when pageSize = 10 and the IM BB defines three services: A, B and C, each of which has 8 clients, the result will be split over 3 pages:
Page 1: Service A, with all of its 8 allowed applications, Service B with 2 allowed applications.
Page 2: Service B, with 6 allowed applications left over, Service C with 4 applications.
Page 3: Service C, with 4 applications.
Implementations can define additional query parameters for result display (such as ordering) and how next page tokens are implemented. It is expected that the next page token is not readable by API users and will be provided verbatim when querying the next page of results.
","operationId":"getAllowed","parameters":[{"in":"query","name":"memberClass","description":"Filter by member class","schema":{"type":"string"},"required":false},{"in":"query","name":"memberCode","schema":{"type":"string"},"description":"Filter by member code","required":false},{"in":"query","name":"applicationId","schema":{"type":"string"},"description":"Filter by application ID","required":false},{"in":"query","name":"serviceId","schema":{"type":"string"},"description":"Filter by service ID","required":false},{"in":"query","name":"pageSize","description":"Number of access rights returned on one page","schema":{"type":"number","default":100},"required":false},{"in":"query","name":"nextPageToken","description":"Handle for the next page, if the result spans multiple pages. If not specified, there are no more results.","schema":{"type":"string"},"required":false}],"responses":{"200":{"description":"OK","content":{"application/json":{"schema":{"allOf":[{"$ref":"#/components/schemas/PaginatedResult"},{"type":"object","properties":{"allowList":{"$ref":"#/components/schemas/ServiceAccessRights"}}}]}}}},"401":{"description":"Caller does not have permissions to view access rights"}}}}}} ``` ## PATCH /api/v1/rights/allow > Allow access to services ```json {"openapi":"3.0.0","info":{"title":"GovStack IM Configuration Management API","version":"1.0.1"},"tags":[{"name":"rights","description":"Access rights management"}],"security":[{"auth":["write:rights"]}],"components":{"securitySchemes":{"auth":{"type":"oauth2","flows":{"authorizationCode":{"authorizationUrl":"https://example.com/oauth/authorize","tokenUrl":"https://example.com/oauth/token","scopes":{"read:conf":"Grants read access to IM configuration","create:conf":"Grants permission to create IM configuration","write:conf":"Grants write access to IM configuration","read:rights":"Grants permission to see what services are allowed","write:rights":"Grants permission to allow/deny IM services"}}}}},"schemas":{"ServiceAccessRights":{"type":"array","items":{"type":"object","required":["memberClass","memberCode","applicationId","serviceId","rights"],"properties":{"memberClass":{"description":"Kind of organisation. Namespace for organisation {code}","type":"string"},"memberCode":{"description":"Registration number (or identifier) of the organisation in {memberClass} namespace","type":"string"},"applicationId":{"type":"string","description":"Name of application"},"serviceId":{"description":"Name of service/API","type":"string"},"rights":{"description":"Applications that are allowed or denied access","type":"array","items":{"$ref":"#/components/schemas/AllowedApplication"}}}}},"AllowedApplication":{"description":"An identified member application that has the permission to access a service.","type":"object","required":["member","application"],"properties":{"memberClass":{"description":"Member or member group identifier to whom access is granted","type":"string"},"memberCode":{"description":"Member or member group identifier to whom access is granted","type":"string"},"applicationId":{"description":"Application of the member that to whom access is granted","type":"string"}}}}},"paths":{"/api/v1/rights/allow":{"patch":{"tags":["rights"],"summary":"Allow access to services","operationId":"setAllowed","requestBody":{"description":"","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ServiceAccessRights"}}},"required":false},"responses":{"200":{"description":"OK"},"401":{"description":"Caller does not have the necessary permissions to update access rights"}}}}}} ``` ## PATCH /api/v1/rights/deny > Deny access to services ```json {"openapi":"3.0.0","info":{"title":"GovStack IM Configuration Management API","version":"1.0.1"},"tags":[{"name":"rights","description":"Access rights management"}],"security":[{"auth":["write:rights"]}],"components":{"securitySchemes":{"auth":{"type":"oauth2","flows":{"authorizationCode":{"authorizationUrl":"https://example.com/oauth/authorize","tokenUrl":"https://example.com/oauth/token","scopes":{"read:conf":"Grants read access to IM configuration","create:conf":"Grants permission to create IM configuration","write:conf":"Grants write access to IM configuration","read:rights":"Grants permission to see what services are allowed","write:rights":"Grants permission to allow/deny IM services"}}}}},"schemas":{"ServiceAccessRights":{"type":"array","items":{"type":"object","required":["memberClass","memberCode","applicationId","serviceId","rights"],"properties":{"memberClass":{"description":"Kind of organisation. Namespace for organisation {code}","type":"string"},"memberCode":{"description":"Registration number (or identifier) of the organisation in {memberClass} namespace","type":"string"},"applicationId":{"type":"string","description":"Name of application"},"serviceId":{"description":"Name of service/API","type":"string"},"rights":{"description":"Applications that are allowed or denied access","type":"array","items":{"$ref":"#/components/schemas/AllowedApplication"}}}}},"AllowedApplication":{"description":"An identified member application that has the permission to access a service.","type":"object","required":["member","application"],"properties":{"memberClass":{"description":"Member or member group identifier to whom access is granted","type":"string"},"memberCode":{"description":"Member or member group identifier to whom access is granted","type":"string"},"applicationId":{"description":"Application of the member that to whom access is granted","type":"string"}}}}},"paths":{"/api/v1/rights/deny":{"patch":{"tags":["rights"],"summary":"Deny access to services","operationId":"setDenied","requestBody":{"description":"","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ServiceAccessRights"}}},"required":false},"responses":{"200":{"description":"OK"},"401":{"description":"Caller does not have the necessary permissions to update access rights"}}}}}} ``` # 9 Internal Workflows This section provides a detailed view of how this Building Block will interact with other Building Blocks to support common use cases. A workflow provides a detailed view of how the Information Mediator Building Block will interact with other building blocks to support common use cases. This section lists workflows that this building block must support. Other workflows may be implemented in addition to those listed. ## 9.1 Service Access Example Sequence diagram for accessing service {% @mermaid/diagram content="sequenceDiagram participant Requestor participant SS1 as IM Security Server of Requestor participant CS as IM Central Server participant CA as CA participant TSA as TSA participant SS2 as IM Security Server of Responder participant Responder Requestor ->> SS1 : Message X SS1 ->> CS : read configuration\* CS -->> SS1 : GovStack configuration SS1 ->> SS1 : find Responder SS1 ->> SS2 : Initiate TLS connection\* SS2 ->> SS1 : Responder cert SS1 ->> CA : OCSP check Responder cert\* CA -->> SS1 : yes/no SS1 ->> SS1 : compose digital document containing "Message X" SS1 ->> TSA : stamp document TSA -->> SS1: timestamp SS1 ->> SS1 : sign document SS1 ->> SS2 : send document over TLS SS2 ->> CA : OCSP check Requestor cert\* CA -->> SS2 : yes/no SS2 ->> SS2 : check document signature SS2 ->> SS2 : write message to log SS2 ->> SS2 : check access rights SS2 ->> Responder : Message X" %} The results of steps marked with '\*' can be cached and reused between calls. In this diagram: 1. Requestor makes a request to the local Information Mediator security server. 2. Local Infirmation Mediator security server finds in the configuration where to send request. 3. The Information Mediator signs & sends the request from application A to the security server for application B. 4. The security server for application B receives the request, validates the signature, and then forwards it to the application/service/endpoint. 5. The application/service/endpoint responds to request. 6. The security server for application B signs the response and sends it back to the security server for application A. 7. The security server for the application A validates the signature and forwards the response to application A. 8. This is all synchronous. Application A’s request is open/unresponded until step 6. ## 9.2 Pub/Sub ### 9.2.1 Registering a Room {% @mermaid/diagram content="sequenceDiagram participant Owner participant SS as IM Security Server participant CS as IM Central Server participant adm as Administrator Owner ->> SS: create Room SS ->> CS: register Room as application CS ->> CS: store request CS -->> SS: registration accepted adm ->> CS: confirm application creation request CS ->> CS: change configuration SS ->> CS: read configuration CS -->> SS: GovStack configuration " %} Room for Pub/Sub must be created before events can be described and processed. Room is created by its owner organization (Member), the organization responsible for the operation of events of a certain type. Creation request should be confirmed by GovStack administrator. ### 9.2.2 Defining event type {% @mermaid/diagram content="sequenceDiagram participant Owner participant SS as IM Security Server participant Server Owner ->> SS: define event type SS ->> Server: get OpenAPI description Server -->> SS: OpenAPI SS ->> SS: create service SS -->> Owner: done" %} Event type corresponds to a service description of a service accepting events for publishing and must be registered before usage. A single room might host multiple event types. E.g. in India, "The Ministry Of Home Affairs" can be the "Owner" of a Room where events of type "emergency" are published by "citizens" or "entities" and entities such as "ambulance services", "Fire-fighting services", Hospitals, etc., can be Subscribers to this Room. In this scenario, an event of type "Fire emergency" can be published by any citizen enrolled in this Room. The Ministry can choose to have one "emergency" Room in each town and enroll Subscribers relevant to respective regions. ### 9.2.3 Subscribing to events of a certain type {% @mermaid/diagram content="sequenceDiagram participant Subscriber participant SS1 as IM Security Server of Subscriber participant SS2 as IM Security Server of Room participant Room opt event type discovery (design time) Subscriber ->> SS1: request list of Members and Rooms SS1 -->> Subscriber: list of Members and Rooms Subscriber ->> Subscriber: chose Room Subscriber ->> SS1: request list of available event types (Services) SS1 ->> SS2: request list of available event types (Services) SS2 -->> SS1: list of event types SS1 -->> Subscriber: list of event types Subscriber ->> SS1: GET description of the event type SS1 ->> SS2: GET description of the event type SS2 -->> SS1: OpenAPI of the event type SS1 -->> Subscriber: OpenAPI of the event type Subscriber ->> SS1: register endpoint for receiving events SS1 -->> Subscriber: done end Subscriber ->> SS1: subscribe to event type SS1 ->> SS2: subscribe to event type SS2 ->> Room: subscribe Room ->> Room: register subscription Room -->> SS2: done SS2 --> SS1: done SS1 --> Subscriber: done" %} 1. The Subscriber requests that events of certain types be delivered to them. 2. The Subscriber specifies the desired delivery mode (push/pull). The Room and Subscriber conclude the delivery contract. Note that: * push delivery mode is when the Room sends events to the Subscriber’s API. In this case, Subscriber may request some QoS/SLA parameters of delivery. * pull delivery mode (OPTIONAL) means that the Room keeps a queue of events and the Subscriber can check that queue. 3. At a minimum, this Building Block MUST provide for a push delivery mode. It may also provide a pull mode based on convenience. Three endpoints to be declared per event type Pub/Sub instance: * an endpoint URL to be registered by every Subscriber on a per-event type basis. * (OPTIONAL) an endpoint URL on the Pub/Sub where all pull requests come from various Subscribers. * an endpoint in the Room to send messages for publication. ### 9.2.4 Event delivery {% @mermaid/diagram content="sequenceDiagram participant Publisher participant Room participant Subscriber Publisher ->> Room: POST event Room ->> Room: store event Room -->> Publisher: event id loop for all Subscribers alt PUSH delivery mode Room ->> Subscriber: POST event Subscriber -->> Room: done else PULL delivery mode Room ->> Room: Store event in Subscriber queue opt asynchronous activity Subscriber ->> Room: GET event of type Room -->> Subscriber: event Subscriber ->> Room: Acknowledge event end end end opt get details Subscriber -->> Publisher: request event details Publisher -->> Subscriber: details end " %} 1. The Publisher and the Room have a data access contract to establish trust for accessing information. * This is akin to the standard trust contract in the Information Mediator, the consumer must have rights to access a certain API. * Publishers have to sign a contract with the Room owner’s consent to gain access to specific Rooms and events. In the contract, Publishers declare what type of message will be published in a given Room. 2. The Publisher generates an event: * The Publisher makes a POST call to the Room service of a particular event type. * The Room stores the event and replies with the event id. Event id is generated by Room or is taken from the original event dataset if provided by the publisher. * The Room can implement a first in, first out (FIFO) event distribution policy. If the Room provides FIFO guarantees, the Room may: * (a) assign a key to the event. If a key attribute is defined in the schema of the event, the value of that attribute will be used. Otherwise, the Room can assign a key at its own discretion. * (b) assign a sequence number to the event. The sequence number must be monotonically increasing for events that share the same key. * The Room sends an immediate acknowledgment to the Publisher. 3. The Room distributes an event asynchronously as follows: * A reference to publisher and event id is added to the event dataset. * For each Subscriber: * (alt) If the mode is ‘push’, make a POST call to Subscriber GovStack service of the event type; * (alt) If the mode is ‘pull’, enqueue an event for request from the Subscriber; * There is a queue of events waiting to be processed per the Subscriber, such that the Subscriber might periodically check to see events waiting in their own queue, process those events, and clear the queue. * A pull mechanism is essential for resilience to network dropouts and traffic load balance at servers and for differentiating urgent/emergency events from normal events (this can be decided during implementation). 4. (OPTIONAL - if mode is PULL) At some moment of time, the Subscriber pulls an event: * The Subscriber makes a GET call to the Room service of the particular event type. * The Subscriber acknowledges receiving of event. 5. (OPTIONAL) The Subscriber requests event details. Some event details may have more restricted regulations for handling and may be not included in event type. In this case, the Subscriber requests these details directly from the publisher by making a GET call to the referenced Publisher with event id as a parameter. * This call implies the existence of an associated contract between the Subscriber and the Publisher. The Information Mediator Building Block creates a log of all messages published and distributed. If an event payload is very large, then it is recommended to just publish the "event" and let Subscribers get full details directly from the Publishers as needed. It will simplify event payloads, and reduce Turnaround time (TAT), storage, and bandwidth significantly. For example, “Clinic System A” wishes to broadcast data about a new birth so that it can be used to trigger asynchronous actions in several other systems. A Service is configured in Clinic System A that makes an HTTP POST request corresponding to the type of event and some valid JSON in the body to the Pub/Sub provider. The provider maintains a list of active “Subscribers” for that event type. Without manipulating the body, that message is effectively forwarded to each subscriber, so that REST services in the “Demographics Tracker”, “Insurance Registry”, and “Early Childhood Education” applications can consume that data and do things based on it. It should be noted that the shape of the payload will be agreed upon beforehand so that the responsibility of being able to “ingest” a “newBirth” payload from Pub/Sub will fall on the REST service provider. Once an event is ingested, the responsibility to deliver the payload lies with the Information Mediator “room” based on the importance given to the event by a Subscriber. Thus, during registration of the Subscriber, parameters such as max queue depth, retries, failsafe mechanisms, and error handling have to be configured by each Subscriber as they subscribe to an event type. # 10 Other Resources This section links to any external documents that may be relevant, such as standards documents or other descriptions of this Building Block that may be useful. ## 10.1 Key Decision Log [A historical log of key decisions regarding this Building Block](https://govstack-global.atlassian.net/wiki/spaces/GH/pages/183369729). ## 10.2 Future Considerations [A list of topics that may be relevant to future versions of this Building Block](https://govstack-global.atlassian.net/wiki/spaces/GH/pages/183500805). # Information Mediator Developed by Aleksander Reitsakas (Aktors, Estonia), Taylor Downs (OpenFn), Dr. P. S. Ramkumar (ITU), and Tõnis Pihlakas (AWS) # 1 Version History The version history table describes the major changes to the specifications between published versions. | Version | Author | Comment | | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | | 0.8 | Aleksander Reitsakas, Taylor Downs, Dr. P. S. Ramkumar | Initial revision | | 0.9 |Aleksander Reitsakas
Technical Reviewers: Tony Shannon, Saša Kovačević, Riham Moawad, Riham Fahmi, Aare Laponin, Manish Srivastava, Palab Saha, Surendra Singh Sucharia, Arvind Gupta, Gayatri. P., Shivank Singh Chauhan, Gavin Lyons
| External links updated | | 1.0 RC1 |Aleksander Reitsakas, Taylor Downs, Dr. P. S. Ramkumar, Aleksa Krolls
Reviewers:
Neil Roy, Aare Lapõnin, Fergal Marrinan, Surendrasingh Sucharia, Amy Darling
Software modules that can be deployed and combined in a standardized manner. Each Building Block is capable of working independently, but they can be combined to do much more. Building Blocks are composable, interoperable software modules that can be used across a variety of use cases. They are standards-based, preferably open-source, and designed for scale. Each Building Block exposes a set of services in the form of REST APIs that can be consumed by other Building Blocks or applications.
Ingress access is access from external applications to GovStack Building Blocks and applications.
Egress access is access from within GovStack Building Blocks and applications to external applications.
| | **Use Case** |A use case is a piece of functionality described as a sequence of actions (steps) to achieve a specific goal in a specific context of usage. E.g., in one use case, the Information Mediator Building Block may be used to let a Building Block access a service provided by another, or in a different use case may be used to relay an event notification from one Building Block to several other Building Blocks via a Publisher-Subscriber (Pub/Sub) model.
Each use case may involve a collection of modules or Building Blocks. A relatively small set of these Building Blocks can be readily applied to a wide variety of use cases in low-resource settings.
| | **API** | An application programming interface ([API](https://en.wikipedia.org/wiki/API)) is a connection between computers or between computer programs. It is a type of software interface, that offers a service to other pieces of software. A document or standard that describes how to build such a connection or interface is called an API specification. A computer system that meets this standard is said to implement or expose an API. The term API may refer either to the specification or to the implementation. | ## 3.5 Service Access | Term | Description | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Member** | A member is an organization that is authorized to communicate via the Information Mediator for a particular GovStack implementation. | | **Application** | An application is a running instance containing one or more Building Block instances and zero or more use case implementations. An application uses an Information Mediator Building Block to communicate with other Building Blocks or applications. An application typically has a single responsibility. | | **Service** | A service is a minimal piece of functionality provided by a Building Block or use case implementation. A service can be local (inside one application) or remote (between different applications). Remote services are consumed using REST protocol and they are described by [OpenAPI](https://github.com/GovStackWorkingGroup/BuildingBlockAPI/tree/main/IM) specification. For example, the “registration” service might be accessed at a particular URL and allow a requester to “register a patient” by sending a POST request with proper patient data. | | **Security Server** | A security server is the main piece of software that is responsible for implementing the “service access” layer of the Information Mediator. This software acts as a gateway and is responsible for mediating requests between various members, applications, and services. It might be not just a single piece, but also a clustered or serverless deployment. | ## 3.6 Pub/Sub | Term | Description | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Publisher** | A Publisher produces events and sends them to rooms. Each event has an event type associated with it. Publishers can produce events of different types. | | **Room** | A Room is a Pub/Sub entity that handles the distribution of events. Each Room has a set of connected event types (E.g., the “birth” Room might contain three event types: “new\_birth”, “birth\_complication”, and “infant\_death”). | | **Subscriber** | A Subscriber can process events of a certain event type. Subscribers are independent of each other and their business logic is different (as rule). Each subscriber processes events from their own perspective. | # 4 Key Digital Functionalities Key Digital Functionalities describe the core (required) functions that this Building Block must be able to perform. The Information Mediator Building Block should provide the following functionalities, organized across 6 different components or layers ### 4.1 Service Access Service access is the facility through which different GovStack Building Blocks and applications may communicate securely with each other. The service access layer should provide an administrative interface that can be used to define users and roles and what access each should have. This layer should also include a registration interface which allows new members to onboard to the Information Mediator. The Service Access layer will also provide functionality to allow or restrict secure access to various resources, route requests to the correct providers, connect service requestors to providers, and enforce access privileges and security policies ### 4.2 Directory Services The Directory service allows applications to discover resources or functionalities that are made available by the system. An application may query the Directory service to discover organizations, services, and APIs that are available to them ### 4.3 Pub/Sub Service The Pub/Sub service is a facility through which applications may publish and subscribe to different events identified by unique message types, enabling more efficient and resilient communication and application design. It should provide broadcast/multicast capabilities to facilitate faster, more resilient application design. ### 4.4 Logging Services The Information Mediator will manage a message log which will provide comprehensive audit trails for all services requests and access. ### 4.5 Monitoring Services The Monitoring services will provide both operational and environmental reporting and dashboards, allowing users to discover the volume of requests to particular services or organizations as well as environmental reports on hardware/network usage. ### 4.6 Scaling/Throughput Services The Information Mediator should support scaling services such as load balancers and allow for the addition of additional servers to manage increased system use. ### Out-of-scope requirements Note that the Information Mediator is not responsible for manipulating the payloads sent to and from various applications—in a sense, it is both the postal service and the roads/bridges/train tracks—but it does not read the contents of your mail. The Information Mediator Building Blocking does not handle communication between human users and applications. The Information Mediator Building Blocking does not handle ingress and egress access from external applications to and from GovStack’s internal components (This may be handled by a public API Gateway in the domain of the Security Building Block). The current specification does not assume the possibility of different Information Mediator implementations working together. In one instance of the GovStack ecosystem, only one implementation of the Information Mediator Building Block can exist. The following requirements have been identified as “out of scope” because they fall into the realm of the Workflow Building Block or the Security Building Block. * Supports the handling of transactions and associated communication errors and exceptions. * Translates data from one format to another, and interoperates with handshake protocols to enable interoperability between different ICT (Information and communication technology) Building Blocks duplications. * Map data structures and fields from the identification system to the registration system and vice versa. * Hold authentication and credentials for each system. * Allow the definition of steps for a particular transaction. * Provide an API for both systems to access – and execute all necessary steps for that transaction (including error handling, retries, and notifications). * Provide an API for external systems to access GovStack Building Blocks. ### **Future Scope** In future iterations of this specification, we may take into consideration more broad API-management standards which include multiple domains, such as those proposed by the United Kingdom's Government in their [Defining an API management strategy](https://www.gov.uk/guidance/defining-an-api-management-strategy) and the section on [Gov.UK’s API Management Strategy Document](#4.3.3.2-gov.uks-api-management-strategy-document). # 5 Cross-Cutting Requirements This section will highlight important requirements or describe any additional cross-cutting requirements that apply to this Building Block. The cross-cutting requirements described in this section are an extension of the cross-cutting requirements defined in the [Architecture Blueprint document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements) and the [Security Requirements](https://govstack.gitbook.io/specification/v/1.0/security-requirements). This section will describe any additional cross-cutting requirements for this Building Block or differences with the Architecture Blueprint. ## **5.1** GET and PUT APIs must be Idempotent (REQUIRED) Application APIs will contain POST endpoints which are not idempotent. GET/PUT/DELETE APIs are idempotent relative to Information Mediator Building Block, but the idempotents of intermediated services depend on the service provider and cannot be specified at this level. ## **5.2** Databases should not Include Business Logic (RECOMMENDED) Databases should not include business logic. We propose that this is a design recommendation which intends to make business logic all live in a clearly visible and accessible location. This may not always be followed, as the benefits of stored procedures (e.g., in reducing database transaction round-trips, etc.) may sometimes outweigh this general design recommendation. ## **5.3** Design for Asynchronous First (RECOMMENDED) The Service Access Layer of the Information Mediator is synchronous first. The Pub/Sub Layer is asynchronous. ## **5.4** Use Standardized Data Formats for Interchange (REQUIRED) Standard formats are used for communication with other Building Blocks. Inside Building Block non-standard protocols can be used. ## **5.5** Use Web Hooks for Callbacks (REQUIRED) OpenAPI spec 3.0 is supported in the first version. Moving to OpenAPI 3.1 is planned for the future. ## Performance Requirements The Performance Requirements specify the basic parameters that an implementing government might use to establish performance requirements for scalability, throughput, and response times when reasonable/necessary. For example: * Minimum Throughput = 100 requests/sec. * Maximum Latency = 1 sec. * Concurrency = 1000 concurrent requests. * All solutions MUST be able to monitor and report, including but not limited to, resource consumption, throughput, latency, average latency, queue depth/backlogs, etc. * All of these indicators MUST be available through an administrative API. * Ideally, all Building Blocks should be able to run a “monitoring agent” which handles reporting out logs, requests, Building Block-specific indicators, etc. to a monitoring service (e.g. [Datadog](https://www.datadoghq.com/)) * The local monitoring agent should be configurable via web interface. * Retries and back-off strategies must be configurable. * Specific “[Scaling/Throughput](https://mediator.govstack.global/information-mediation-1.0/pages/OyfUt4flSKVkHBSwoOcO#6.6-scaling-throughput)” requirements are in the functional requirements section. # 6 Functional Requirements This section lists the technical capabilities of this Building Block. The functional requirements section lists the technical capabilities that this Building Block should have. These requirements should be sufficient to deliver all functionality that is listed in the [Key Digital Functionalities](/information-mediation-1.0/4-key-digital-functionalities) section. The Information Mediator is used only for interaction between Applications/Building Blocks for the transfer of data at an API level only. Details in the [Architecture and Nonfunctional Requirements document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements/6-onboarding). These functional requirements do not define specific APIs, they provide a list of information about functionality that must be implemented within the Building Block. Detailed design and feature lists of these blocks can be customized by developers to optimally match specific target implementation needs. ## **6.1** Service Access #### **6.1.1 Administrative Interface** * There should be different types of administrative roles that provide different levels of access (RECOMMENDED) * There should be administrative roles on different levels of the ecosystem (RECOMMENDED): * global administrator role of the whole ecosystem, * local administrator role of Member, * an administrative role for configuring an Application. * There should be an ecosystem-level administrative interface that allows an Information Mediator administrator (with appropriate authentication) to register/deregister members and applications (RECOMMENDED) * There should be a security-server-level interface that allows an administrator for a single security server to manage members, applications, and services that live under a single security server (RECOMMENDED) * Open IAM (or another Identity and Access Management Solution) must be able to create/edit/delete admin users for the Information Mediator interface. [Security Building Block specification 1.0.1 section 6-1](https://govstack.gitbook.io/specification/v/1.0/security-requirements/6-security-building-block-modules) (REQUIRED) #### **6.1.2 Registration** (REQUIRED) * Registering a member/organization requires: * That the new member provides details for registration. * That the administrator verifies and accepts the request for registration. * Registering an application requires: * That the new member provides details for registration. * That the administrator verifies and accepts the request for registration. * Registering a service requires (in terms of API endpoints): * That the owner of the application provides: * OpenAPI 3.0 specification document (i.e., includes base paths like “/api/patients”, “/api/visits”, etc.). * Description. * List of enabled endpoints of OpenAPI specifications (the requirement is that we can enable/disable endpoints). * Managing a list of allowed consumers for services requires: * Application developers may access the directory service (described below) to select the services that they want to consume (OPTIONAL) * The request/approval/addition of allowed consumers process is a business-first process with manual steps (The hard part is negotiating data sharing agreements and signing contracts when that is completed the Information Mediator administrator can easily modify the allowed consumers' list manually) * An application must specify which member/application/service they want to access * The provider of that service must decide if the consumer is allowed to * Once approved, the requesting application will be added to the list of allowed applications for the requested service #### **6.1.3 Accessing Services** * To make a request to another service via the Information Mediator, an application must(REQUIRED): * Using REST, make a valid HTTPS request to the local Information Mediator security server with headers that identifies itself at the application level. * The components of the request must be (with color code): * Security server URL; * API version; * Instance (e.g., Country); * Domain of member; * Member (e.g., Ministry of X); * Application; * Service (OpenAPI file); * Path * Endpoint. * Query parameters. **Example Only**\ *Sample GET Request:*\ url-of-local-information-mediator-security-server/r1/INDIA/GOV/ministry-of-agriculture-karnataka/market-linkages-app/inventory-service/v1/check-level/apples?fresh=true\ *Response*: { data: 7 }\ \ *Sample POST Request:*\ url-of-local-information-mediator-security-server/r1/INDIA/PVT/tata-buyers-corp-karnataka/small-farmer-buyers-app/inventory-service/v1/supply/apples\ *With body*: {“qtyAvailable”: 4}\ *Response*: { result: “Stock level report created.” }\ \ *How to interpret the above request paths:*\ SECURITY-SERVER-URL/r1/INSTANCE/DOMAIN/MEMBER/APPLICATION/SERVICE/PATH * Note that all applications are making requests to the security server, which runs over the private network segment, rather than making requests to other applications directly over the public internet. (This is one of the main points of the security server and Information Mediator architecture.) ## 6.2 Directory Services * At development time, to see which resources are available on GovStack, the administrator/developer of application A may send requests to the security server to see(OPTIONAL): * organizations with descriptions; * services (either all services, or services that the requester is authorized to access); * an OpenAPI specification, detailing the endpoints and requirements for the service/API. * A view layer allowing for easy exploration of ALL clients, applications, and services should be provided. (Note that, “under the hood”, this layer may make use of the APIs described above or be implemented via a separate API.)(RECOMMENDED) ## 6.3 Pub/Sub Service #### 6.3.1 Defining Rooms and Event Types * The Pub/Sub layer must provide a mechanism for registering Rooms (REQUIRED) * The Pub/Sub layer must provide a mechanism for registering event types (REQUIRED) * For each event type, an OpenAPI description of a service with JSON schema for the required payload shape must be defined to create an event of the type (REQUIRED) * An interface for registering and viewing event types must be provided (REQUIRED) #### **6.3.2 Publishing a Message** * An application must be able to make a POST request with a valid JSON payload as the body and specify the message type to a Room provided by the Pub/Sub layer service (REQUIRED) #### 6.3.3 Delivering Messages * Published messages should be delivered to all subscribers (RECOMMENDED) * Published messages should be stored so that delivery may be done and retried asynchronously, e.g. if certain Subscribers are offline (RECOMMENDED) * Subscribers should be able to configure their retry strategies, overriding the default exponential backoff on retriable errors ([Google Pub/Sub example](https://cloud.google.com/pubsub/docs/handling-failures)) * If an active subscription exists but an error is received when forwarding a message to that service, the Pub/Sub layer should retry N times with a standard backoff. (An exponential backoff may be the default approach.) (RECOMMENDED) * The backoff and retry strategy should be configurable by an administrator with access to the Pub/Sub layer application * If a message cannot be delivered the Pub/Sub layer should drop that message (RECOMMENDED) #### 6.3.4 Managing Subscriptions * The Pub/Sub layer should provide an admin user interface to help create/manage subscriptions with the data below (RECOMMENDED) * The Pub/Sub layer should allow an administrator to view a list of active subscriptions (RECOMMENDED) * For registration via API, an application must be able to make a POST request to a service exposed by the Pub/Sub layer which defines which endpoints certain event types should be sent to (REQUIRED) #### 6.3.5 Event Logging and Audit Trail Generation * All events received and delivered must have a unique ID (REQUIRED) * All events received by the Pub/Sub layer must be logged or added to a log sync and those log entries must contain event metadata including the sender, timestamp, and event type, but may not include the event payload (REQUIRED) * All event delivery attempts must be logged or added to a log sync (REQUIRED) * For every event message received, the Information Mediator sends back an acknowledgment with the ID of the event to the respective Publisher (REQUIRED) * There must be a possibility to search and view the status of messages, for example, a message with type X to seven subscribers had been successfully delivered to all seven (REQUIRED) ## 6.4 Logging Services * The Information Mediator Building Block maintains a message log (REQUIRED) * The purpose of the message log is to provide means to prove the reception of a regular request or response message to a third party. Messages exchanged between Information Mediator Building Block are signed and encrypted. For every regular request and response, the security server produces a completely signed, and timestamped document. At a minimum, the log must store metadata that identifies a specific message, the status of transaction carried out on that message by Information Mediator Building Block, along with source ID and date time stamp. * The Information Mediator Building Block has full audit trail capabilities (REQUIRED) * The Information Mediator Building Block keeps an audit log. The audit log events are generated by the user interface when the administrator changes the system's state or configuration. The administrator's actions are logged regardless of whether the outcome was a success or a failure. The system must be capable of emitting statistical reports for a given organization, application, service, or consumer and status. ## 6.5 Monitoring Services * Operational monitoring (REQUIRED) * Operational monitoring provides details about the requested exchange, such as the ID-s of the client and the service, various attributes of the message read from the message header, request and response timestamps, sizes, etc., but not the actual payload of messages. * The operational monitoring daemon collects and shares operational monitoring data of the Information Mediator Building Block as part of request exchange, shares this data, and calculates and shares health statistics (the timestamps and the number of successful/unsuccessful requests, various metrics of the duration and message size of the requests, etc.). * The operational monitoring daemon makes operational and health data available to the owner of the security server, regular clients, and the central monitoring client via the security server. (For example, local health data may be made available for external monitoring systems.) * The owner of the security server and the central monitoring client are able to query the records of all clients. For a regular client, only the records associated with that client are available. * Environmental monitoring (REQUIRED) * Environmental monitoring provides details of the security servers such as operating system, memory, disk space, CPU load, traffic load, running processes, installed packages, etc. in a chosen date range. * Environmental monitoring provides a standard endpoint that can be accessed with a client (e.g. Java's console application if using Java Management Extensions). * It is possible to limit what allowed non-owners can request via environmental monitoring data requests. The security server owner will always get the full data set as requested. ## 6.6 Scaling/Throughput Services * The Information Mediator Building Block should support provider-side high availability setup if needed (RECOMMENDED) * Busy production systems may need a scalable performance in addition to high availability. The Information Mediator Building Block supports external load balancing mechanisms to address both of these problems simultaneously. A load balancer is added in front of a security server cluster to route the requests based on a selected algorithm (OPTIONAL) * The team in charge of deploying the security server application on their hardware must consider the network infrastructure including a load balancer, etc. The requirements for network infrastructure must be handled/defined in a (yet to exist) “Network Building Block”(REQUIRED) # 7 Data Structures This section provides information on the core data structures/data models that are used by this Building Block. The resource model shows the relationship between data objects that are used by the Information Mediator Building Block. The data elements provide detail for the resource model. All data element schemas can be viewed, commented on, and modified in the [schemas section of the Information Mediator Building Block GitHub repository](https://github.com/GovStackWorkingGroup/bb-information-mediator/tree/1.0-QA/schemas). ## 7.1 Resource Model The Mediator Building Block key element is Service. The Service is used by a consuming Building Block or an application and offered by a provider Building Block or Application. Both Provider and Consumer must be Members of the Mediator Building Block. Members of the Information Mediator Building Block can be an organization (governmental or not, business or not) or a person (citizen as a rule).  To become a Member of Mediator Building Block participants must fulfill declared requirements and apply for onboarding. In the process of onboarding a Member is registered with Mediator Building Block and gets credentials to connect to Mediator Building Block. Normally it is done the way that a Member provides a certificate of recognized Certification Authority (CA), and requests signed with this certificate are considered legitimate requests of the Member. A member entity can access the Pub/Sub configuration and register a room to publish its own event type as a publisher through its own admin. Members can browse a directory of Services available in the Mediator Building Block. Each service is described in OpenAPI. ## 7.2 Data Elements ### 7.2.1 Member **Fields:** | Name | Type | Description | | ------------ | ----------- | -------------------------------------------------------- | | class | enumerator | namespace for member\_code, e.g. GOV or COM | | code | string | unique id of member in registry defined by member\_class | | signing\_key | string | name of key used to sign on behalf of Member | | certificate | certificate | certificate of member issued by trusted CA | Schema reference [member.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/1.0-QA/schemas/member.json) ### 7.2.2 Application **Fields:** | Name | Type | Description | | ---------------- | ----------- | ------------------------------------------- | | code | string | unique name of application in member scope | | connection\_type | string | protocol used for connection: http or https | | certificate | certificate | TLS certificate | Schema reference [application.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/1.0-QA/schemas/application.json) ### 7.2.3 Service **Fields:** | Name | Type | Description | | ---------------- | ------ | ---------------------------------------------- | | code | string | unique name of OpenAPI in scope of application | | description\_url | url | location of OpenAPI service description | | service\_url | url | service provisioning network address | | ACL | object | description of access rights | Schema reference [service.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/1.0-QA/schemas/service.json) ## **7.3 PubSub Layer** ### 7.3.1 Resource Model The Resource Model is an extension of the Access Layer model:  ### 7.3.2 Data Elements #### 7.3.2.1 Event An event is a message—a set of data sent to a topic. Each event has an id. The event corresponds to the message. Data elements of an event are described by event type OpenAPI description. #### 7.3.2.2 Event Type An event type is schema definition for an event. Each event type is owned by a Room of a certain authority. (E.g. the Ministry of Health might own Room with the “new\_birth” event type and define its schema.) #### 7.3.2.3 Publisher A candidate application playing the role of **IM-Publisher** must be able to emit events to a specific Room.| Data Element | Default format | Description |
|---|---|---|
| id | string | application id of this publisher |
| name | string | OPTIONAL |
Aleksander Reitsakas
Technical Reviewers: Tony Shannon, Saša Kovačević, Riham Moawad, Riham Fahmi, Aare Laponin, Manish Srivastava, Palab Saha, Surendra Singh Sucharia, Arvind Gupta, Gayatri. P., Shivank Singh Chauhan, Gavin Lyons
| External links updated | | 1.0 RC1 |Aleksander Reitsakas, Taylor Downs, Dr. P. S. Ramkumar, Aleksa Krolls
Reviewers:
Neil Roy, Aare Lapõnin, Fergal Marrinan, Surendrasingh Sucharia, Amy Darling
Software modules that can be deployed and combined in a standardized manner. Each Building Block is capable of working independently, but they can be combined to do much more. Building Blocks are composable, interoperable software modules that can be used across a variety of use cases. They are standards-based, preferably open-source, and designed for scale. Each Building Block exposes a set of services in the form of REST APIs that can be consumed by other Building Blocks or applications.
Ingress access is access from external applications to GovStack Building Blocks and applications.
Egress access is access from within GovStack Building Blocks and applications to external applications.
| | **Use Case** |A use case is a piece of functionality described as a sequence of actions (steps) to achieve a specific goal in a specific context of usage. E.g., in one use case, the Information Mediator Building Block may be used to let a Building Block access a service provided by another, or in a different use case may be used to relay an event notification from one Building Block to several other Building Blocks via a Publisher-Subscriber (Pub/Sub) model.
Each use case may involve a collection of modules or Building Blocks. A relatively small set of these Building Blocks can be readily applied to a wide variety of use cases in low-resource settings.
| | **API** | An application programming interface ([API](https://en.wikipedia.org/wiki/API)) is a connection between computers or between computer programs. It is a type of software interface, that offers a service to other pieces of software. A document or standard that describes how to build such a connection or interface is called an API specification. A computer system that meets this standard is said to implement or expose an API. The term API may refer either to the specification or to the implementation. | ## 3.5 Service Access | Term | Description | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Member** | A member is an organization that is authorized to communicate via the Information Mediator for a particular GovStack implementation. | | **Application** | An application is a running instance containing one or more Building Block instances and zero or more use case implementations. An application uses an Information Mediator Building Block to communicate with other Building Blocks or applications. An application typically has a single responsibility. | | **Service** | A service is a minimal piece of functionality provided by a Building Block or use case implementation. A service can be local (inside one application) or remote (between different applications). Remote services are consumed using REST protocol and they are described by [OpenAPI](https://github.com/GovStackWorkingGroup/BuildingBlockAPI/tree/main/IM) specification. For example, the “registration” service might be accessed at a particular URL and allow a requester to “register a patient” by sending a POST request with proper patient data. | | **Security Server** | A security server is the main piece of software that is responsible for implementing the “service access” layer of the Information Mediator. This software acts as a gateway and is responsible for mediating requests between various members, applications, and services. It might be not just a single piece, but also a clustered or serverless deployment. | ## 3.6 Pub/Sub | Term | Description | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Publisher** | A Publisher produces events and sends them to rooms. Each event has an event type associated with it. Publishers can produce events of different types. | | **Room** | A Room is a Pub/Sub entity that handles the distribution of events. Each Room has a set of connected event types (E.g., the “birth” Room might contain three event types: “new\_birth”, “birth\_complication”, and “infant\_death”). | | **Subscriber** | A Subscriber can process events of a certain event type. Subscribers are independent of each other and their business logic is different (as rule). Each subscriber processes events from their own perspective. | # 4 Key Digital Functionalities Key Digital Functionalities describe the core (required) functions that this Building Block must be able to perform. The Information Mediator Building Block should provide the following functionalities, organized across 6 different components or layers ### 4.1 Service Access Service access is the facility through which different GovStack Building Blocks and applications may communicate securely with each other. The service access layer should provide an administrative interface that can be used to define users and roles and what access each should have. This layer should also include a registration interface which allows new members to onboard to the Information Mediator. The Service Access layer will also provide functionality to allow or restrict secure access to various resources, route requests to the correct providers, connect service requestors to providers, and enforce access privileges and security policies ### 4.2 Directory Services The Directory service allows applications to discover resources or functionalities that are made available by the system. An application may query the Directory service to discover organizations, services, and APIs that are available to them ### 4.3 Pub/Sub Service The Pub/Sub service is a facility through which applications may publish and subscribe to different events identified by unique message types, enabling more efficient and resilient communication and application design. It should provide broadcast/multicast capabilities to facilitate faster, more resilient application design. ### 4.4 Logging Services The Information Mediator will manage a message log which will provide comprehensive audit trails for all services requests and access. ### 4.5 Monitoring Services The Monitoring services will provide both operational and environmental reporting and dashboards, allowing users to discover the volume of requests to particular services or organizations as well as environmental reports on hardware/network usage. ### 4.6 Scaling/Throughput Services The Information Mediator should support scaling services such as load balancers and allow for the addition of additional servers to manage increased system use. ### Out-of-scope requirements Note that the Information Mediator is not responsible for manipulating the payloads sent to and from various applications—in a sense, it is both the postal service and the roads/bridges/train tracks—but it does not read the contents of your mail. The Information Mediator Building Blocking does not handle communication between human users and applications. The Information Mediator Building Blocking does not handle ingress and egress access from external applications to and from GovStack’s internal components (This may be handled by a public API Gateway in the domain of the Security Building Block). The current specification does not assume the possibility of different Information Mediator implementations working together. In one instance of the GovStack ecosystem, only one implementation of the Information Mediator Building Block can exist. The following requirements have been identified as “out of scope” because they fall into the realm of the Workflow Building Block or the Security Building Block. * Supports the handling of transactions and associated communication errors and exceptions. * Translates data from one format to another, and interoperates with handshake protocols to enable interoperability between different ICT (Information and communication technology) Building Blocks duplications. * Map data structures and fields from the identification system to the registration system and vice versa. * Hold authentication and credentials for each system. * Allow the definition of steps for a particular transaction. * Provide an API for both systems to access – and execute all necessary steps for that transaction (including error handling, retries, and notifications). * Provide an API for external systems to access GovStack Building Blocks. ### **Future Scope** In future iterations of this specification, we may take into consideration more broad API-management standards which include multiple domains, such as those proposed by the United Kingdom's Government in their [Defining an API management strategy](https://www.gov.uk/guidance/defining-an-api-management-strategy) and the section on [Gov.UK’s API Management Strategy Document](#4.3.3.2-gov.uks-api-management-strategy-document). # 5 Cross-Cutting Requirements This section will highlight important requirements or describe any additional cross-cutting requirements that apply to this Building Block. The cross-cutting requirements described in this section are an extension of the cross-cutting requirements defined in the [Architecture Blueprint document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements) and the [Security Requirements](https://govstack.gitbook.io/specification/v/1.0/security-requirements). This section will describe any additional cross-cutting requirements for this Building Block or differences with the Architecture Blueprint. ## **5.1** GET and PUT APIs must be Idempotent (REQUIRED) Application APIs will contain POST endpoints which are not idempotent. GET/PUT/DELETE APIs are idempotent relative to Information Mediator Building Block, but the idempotents of intermediated services depend on the service provider and cannot be specified at this level. ## **5.2** Databases should not Include Business Logic (RECOMMENDED) Databases should not include business logic. We propose that this is a design recommendation which intends to make business logic all live in a clearly visible and accessible location. This may not always be followed, as the benefits of stored procedures (e.g., in reducing database transaction round-trips, etc.) may sometimes outweigh this general design recommendation. ## **5.3** Design for Asynchronous First (RECOMMENDED) The Service Access Layer of the Information Mediator is synchronous first. The Pub/Sub Layer is asynchronous. ## **5.4** Use Standardized Data Formats for Interchange (REQUIRED) Standard formats are used for communication with other Building Blocks. Inside Building Block non-standard protocols can be used. ## **5.5** Use Web Hooks for Callbacks (REQUIRED) OpenAPI spec 3.0 is supported in the first version. Moving to OpenAPI 3.1 is planned for the future. ## Performance Requirements The Performance Requirements specify the basic parameters that an implementing government might use to establish performance requirements for scalability, throughput, and response times when reasonable/necessary. For example: * Minimum Throughput = 100 requests/sec. * Maximum Latency = 1 sec. * Concurrency = 1000 concurrent requests. * All solutions MUST be able to monitor and report, including but not limited to, resource consumption, throughput, latency, average latency, queue depth/backlogs, etc. * All of these indicators MUST be available through an administrative API. * Ideally, all Building Blocks should be able to run a “monitoring agent” which handles reporting out logs, requests, Building Block-specific indicators, etc. to a monitoring service (e.g. [Datadog](https://www.datadoghq.com/)) * The local monitoring agent should be configurable via web interface. * Retries and back-off strategies must be configurable. * Specific “[Scaling/Throughput](https://mediator.govstack.global/information-mediation-1.0.1/pages/PrLXoPRkL6cmBw9LkMFE#6.6-scaling-throughput)” requirements are in the functional requirements section. # 6 Functional Requirements This section lists the technical capabilities of this Building Block. The functional requirements section lists the technical capabilities that this Building Block should have. These requirements should be sufficient to deliver all functionality that is listed in the [Key Digital Functionalities](/information-mediation-1.0.1/4-key-digital-functionalities) section. The Information Mediator is used only for interaction between Applications/Building Blocks for the transfer of data at an API level only. Details in the [Architecture and Nonfunctional Requirements document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements/6-onboarding). These functional requirements do not define specific APIs, they provide a list of information about functionality that must be implemented within the Building Block. Detailed design and feature lists of these blocks can be customized by developers to optimally match specific target implementation needs. ## **6.1** Service Access #### **6.1.1 Administrative Interface** * There should be different types of administrative roles that provide different levels of access (RECOMMENDED) * There should be administrative roles on different levels of the ecosystem (RECOMMENDED): * global administrator role of the whole ecosystem, * local administrator role of Member, * an administrative role for configuring an Application. * There should be an ecosystem-level administrative interface that allows an Information Mediator administrator (with appropriate authentication) to register/deregister members and applications (RECOMMENDED) * There should be a security-server-level interface that allows an administrator for a single security server to manage members, applications, and services that live under a single security server (RECOMMENDED) * Open IAM (or another Identity and Access Management Solution) must be able to create/edit/delete admin users for the Information Mediator interface. [Security Building Block specification 1.0.1 section 6-1](https://govstack.gitbook.io/specification/v/1.0/security-requirements/6-security-building-block-modules) (REQUIRED) #### **6.1.2 Registration** (REQUIRED) * Registering a member/organization requires: * That the new member provides details for registration. * That the administrator verifies and accepts the request for registration. * Registering an application requires: * That the new member provides details for registration. * That the administrator verifies and accepts the request for registration. * Registering a service requires (in terms of API endpoints): * That the owner of the application provides: * OpenAPI 3.0 specification document (i.e., includes base paths like “/api/patients”, “/api/visits”, etc.). * Description. * List of enabled endpoints of OpenAPI specifications (the requirement is that we can enable/disable endpoints). * Managing a list of allowed consumers for services requires: * Application developers may access the directory service (described below) to select the services that they want to consume (OPTIONAL) * The request/approval/addition of allowed consumers process is a business-first process with manual steps (The hard part is negotiating data sharing agreements and signing contracts when that is completed the Information Mediator administrator can easily modify the allowed consumers' list manually) * An application must specify which member/application/service they want to access * The provider of that service must decide if the consumer is allowed to * Once approved, the requesting application will be added to the list of allowed applications for the requested service #### **6.1.3 Accessing Services** * To make a request to another service via the Information Mediator, an application must(REQUIRED): * Using REST, make a valid HTTPS request to the local Information Mediator security server with headers that identifies itself at the application level. * The components of the request must be (with color code): * Security server URL; * API version; * Instance (e.g., Country); * Domain of member; * Member (e.g., Ministry of X); * Application; * Service (OpenAPI file); * Path * Endpoint. * Query parameters. **Example Only**\ *Sample GET Request:*\ url-of-local-information-mediator-security-server/r1/INDIA/GOV/ministry-of-agriculture-karnataka/market-linkages-app/inventory-service/v1/check-level/apples?fresh=true\ *Response*: { data: 7 }\ \ *Sample POST Request:*\ url-of-local-information-mediator-security-server/r1/INDIA/PVT/tata-buyers-corp-karnataka/small-farmer-buyers-app/inventory-service/v1/supply/apples\ *With body*: {“qtyAvailable”: 4}\ *Response*: { result: “Stock level report created.” }\ \ *How to interpret the above request paths:*\ SECURITY-SERVER-URL/r1/INSTANCE/DOMAIN/MEMBER/APPLICATION/SERVICE/PATH * Note that all applications are making requests to the security server, which runs over the private network segment, rather than making requests to other applications directly over the public internet. (This is one of the main points of the security server and Information Mediator architecture.) ## 6.2 Directory Services * At development time, to see which resources are available on GovStack, the administrator/developer of application A may send requests to the security server to see(OPTIONAL): * organizations with descriptions; * services (either all services, or services that the requester is authorized to access); * an OpenAPI specification, detailing the endpoints and requirements for the service/API. * A view layer allowing for easy exploration of ALL clients, applications, and services should be provided. (Note that, “under the hood”, this layer may make use of the APIs described above or be implemented via a separate API.)(RECOMMENDED) ## 6.3 Pub/Sub Service #### 6.3.1 Defining Rooms and Event Types * The Pub/Sub layer must provide a mechanism for registering Rooms (REQUIRED) * The Pub/Sub layer must provide a mechanism for registering event types (REQUIRED) * For each event type, an OpenAPI description of a service with JSON schema for the required payload shape must be defined to create an event of the type (REQUIRED) * An interface for registering and viewing event types must be provided (REQUIRED) #### **6.3.2 Publishing a Message** * An application must be able to make a POST request with a valid JSON payload as the body and specify the message type to a Room provided by the Pub/Sub layer service (REQUIRED) #### 6.3.3 Delivering Messages * Published messages should be delivered to all subscribers (RECOMMENDED) * Published messages should be stored so that delivery may be done and retried asynchronously, e.g. if certain Subscribers are offline (RECOMMENDED) * Subscribers should be able to configure their retry strategies, overriding the default exponential backoff on retriable errors ([Google Pub/Sub example](https://cloud.google.com/pubsub/docs/handling-failures)) * If an active subscription exists but an error is received when forwarding a message to that service, the Pub/Sub layer should retry N times with a standard backoff. (An exponential backoff may be the default approach.) (RECOMMENDED) * The backoff and retry strategy should be configurable by an administrator with access to the Pub/Sub layer application * If a message cannot be delivered the Pub/Sub layer should drop that message (RECOMMENDED) #### 6.3.4 Managing Subscriptions * The Pub/Sub layer should provide an admin user interface to help create/manage subscriptions with the data below (RECOMMENDED) * The Pub/Sub layer should allow an administrator to view a list of active subscriptions (RECOMMENDED) * For registration via API, an application must be able to make a POST request to a service exposed by the Pub/Sub layer which defines which endpoints certain event types should be sent to (REQUIRED) #### 6.3.5 Event Logging and Audit Trail Generation * All events received and delivered must have a unique ID (REQUIRED) * All events received by the Pub/Sub layer must be logged or added to a log sync and those log entries must contain event metadata including the sender, timestamp, and event type, but may not include the event payload (REQUIRED) * All event delivery attempts must be logged or added to a log sync (REQUIRED) * For every event message received, the Information Mediator sends back an acknowledgment with the ID of the event to the respective Publisher (REQUIRED) * There must be a possibility to search and view the status of messages, for example, a message with type X to seven subscribers had been successfully delivered to all seven (REQUIRED) ## 6.4 Logging Services * The Information Mediator Building Block maintains a message log (REQUIRED) * The purpose of the message log is to provide means to prove the reception of a regular request or response message to a third party. Messages exchanged between Information Mediator Building Block are signed and encrypted. For every regular request and response, the security server produces a completely signed, and timestamped document. At a minimum, the log must store metadata that identifies a specific message, the status of transaction carried out on that message by Information Mediator Building Block, along with source ID and date time stamp. * The Information Mediator Building Block has full audit trail capabilities (REQUIRED) * The Information Mediator Building Block keeps an audit log. The audit log events are generated by the user interface when the administrator changes the system's state or configuration. The administrator's actions are logged regardless of whether the outcome was a success or a failure. The system must be capable of emitting statistical reports for a given organization, application, service, or consumer and status. ## 6.5 Monitoring Services * Operational monitoring (REQUIRED) * Operational monitoring provides details about the requested exchange, such as the ID-s of the client and the service, various attributes of the message read from the message header, request and response timestamps, sizes, etc., but not the actual payload of messages. * The operational monitoring daemon collects and shares operational monitoring data of the Information Mediator Building Block as part of request exchange, shares this data, and calculates and shares health statistics (the timestamps and the number of successful/unsuccessful requests, various metrics of the duration and message size of the requests, etc.). * The operational monitoring daemon makes operational and health data available to the owner of the security server, regular clients, and the central monitoring client via the security server. (For example, local health data may be made available for external monitoring systems.) * The owner of the security server and the central monitoring client are able to query the records of all clients. For a regular client, only the records associated with that client are available. * Environmental monitoring (REQUIRED) * Environmental monitoring provides details of the security servers such as operating system, memory, disk space, CPU load, traffic load, running processes, installed packages, etc. in a chosen date range. * Environmental monitoring provides a standard endpoint that can be accessed with a client (e.g. Java's console application if using Java Management Extensions). * It is possible to limit what allowed non-owners can request via environmental monitoring data requests. The security server owner will always get the full data set as requested. ## 6.6 Scaling/Throughput Services * The Information Mediator Building Block should support provider-side high availability setup if needed (RECOMMENDED) * Busy production systems may need a scalable performance in addition to high availability. The Information Mediator Building Block supports external load balancing mechanisms to address both of these problems simultaneously. A load balancer is added in front of a security server cluster to route the requests based on a selected algorithm (OPTIONAL) * The team in charge of deploying the security server application on their hardware must consider the network infrastructure including a load balancer, etc. The requirements for network infrastructure must be handled/defined in a (yet to exist) “Network Building Block”(REQUIRED) # 7 Data Structures This section provides information on the core data structures/data models that are used by this Building Block. The resource model shows the relationship between data objects that are used by the Information Mediator Building Block. The data elements provide detail for the resource model. All data element schemas can be viewed, commented on, and modified in the [schemas section of the Information Mediator Building Block GitHub repository](https://github.com/GovStackWorkingGroup/bb-information-mediator/tree/1.0-QA/schemas). ## 7.1 Resource Model The Mediator Building Block key element is Service. The Service is used by a consuming Building Block or an application and offered by a provider Building Block or Application. Both Provider and Consumer must be Members of the Mediator Building Block. Members of the Information Mediator Building Block can be an organization (governmental or not, business or not) or a person (citizen as a rule).  To become a Member of Mediator Building Block participants must fulfill declared requirements and apply for onboarding. In the process of onboarding a Member is registered with Mediator Building Block and gets credentials to connect to Mediator Building Block. Normally it is done the way that a Member provides a certificate of recognized Certification Authority (CA), and requests signed with this certificate are considered legitimate requests of the Member. A member entity can access the Pub/Sub configuration and register a room to publish its own event type as a publisher through its own admin. Members can browse a directory of Services available in the Mediator Building Block. Each service is described in OpenAPI. ## 7.2 Data Elements ### 7.2.1 Member **Fields:** | Name | Type | Description | | ------------ | ----------- | -------------------------------------------------------- | | class | enumerator | namespace for member\_code, e.g. GOV or COM | | code | string | unique id of member in registry defined by member\_class | | signing\_key | string | name of key used to sign on behalf of Member | | certificate | certificate | certificate of member issued by trusted CA | Schema reference [member.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/1.0.1/schemas/member.json) ### 7.2.2 Application **Fields:** | Name | Type | Description | | ---------------- | ----------- | ------------------------------------------- | | code | string | unique name of application in member scope | | connection\_type | string | protocol used for connection: http or https | | certificate | certificate | TLS certificate | Schema reference [application.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/1.0.1/schemas/application.json) ### 7.2.3 Service **Fields:** | Name | Type | Description | | ---------------- | ------ | ---------------------------------------------- | | code | string | unique name of OpenAPI in scope of application | | description\_url | url | location of OpenAPI service description | | service\_url | url | service provisioning network address | | ACL | object | description of access rights | Schema reference [service.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/1.0.1/schemas/service.json) ## **7.3 PubSub Layer** ### 7.3.1 Resource Model The Resource Model is an extension of the Access Layer model:  ### 7.3.2 Data Elements #### 7.3.2.1 Event An event is a message—a set of data sent to a topic. Each event has an id. The event corresponds to the message. Data elements of an event are described by event type OpenAPI description. #### 7.3.2.2 Event Type An event type is schema definition for an event. Each event type is owned by a Room of a certain authority. (E.g. the Ministry of Health might own Room with the “new\_birth” event type and define its schema.) #### 7.3.2.3 Publisher A candidate application playing the role of **IM-Publisher** must be able to emit events to a specific Room. | Data Element | Default format | Description | | ------------ | -------------- | -------------------------------- | | id | string | application id of this publisher | | name | string | OPTIONAL | #### **7.3.2.4 Room** *(N.B., this is often called a "topic" and we may shift to that in later versions.)* A candidate application playing the role of **IM-Room** must handle the distribution of events. Each Room has a set of connected event types (e.g., the “birth” room might contain three event types: “new\_birth”, “birth\_complication”, and “infant\_death”). A room is located in the member’s local Information Mediator Building Block implementation and the member is responsible for all types of events in that particular room. | Data Element | Default format | Description | | ------------ | -------------- | --------------------------- | | id | string | application id of this room | | name | string | OPTIONAL | #### 7.3.2.5 Subscriber A candidate application playing the role of **IM-Subscriber** must be able to process events. It is done by defining a Service that will be called by Room to deliver an event. Subscribers are independent of each other and their business logic differs (as a rule). Each subscriber processes events from their own perspective. | Data Element | Default format | Description | | ------------ | -------------- | --------------------------------- | | id | string | application id of this subscriber | | name | string | OPTIONAL | #### 7.3.2.6 Subscription | Data Element | Default format | Description | | -------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | id | string | subscription id | | room\_id | string | room id | | subscriber\_id | string | subscriber id | | event\_type | string | A filter expression that allows a subscriber to subscribe to only certain message\_types that are published to the room they're subscribed to. | | mode | enum | delivery mode | | details | object | details of delivery, like time to live, repetition policy, etc. | Schema reference [broadcast.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/1.0.1/schemas/broadcast.json) # 8 Service APIs This section provides a reference for APIs that should be implemented by this Building Block. The [GovStack non-functional requirements document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements/6-onboarding) provides additional information on how 'adaptors' may be used to translate an existing API to the patterns described here. This section also provides guidance on how candidate products are tested and how GovStack validates a product's API against the API specifications defined here. The tests for the Information Mediator Building Block can be found in [this GitHub repository](https://github.com/GovStackWorkingGroup/bb-information-mediator/tree/main/test/openAPI). The majority of functions provided by the Information Mediator Building Block are either defined in the “service access flow” or configured by the administrator via the web User Interface. There is, however, a “Directory Service” which can provide listings of clients, methods, and available API specifications for services on the Information Mediator. The directory is managed by admins of members. The directory service centralizes and offers knowledge of all enrolled members and their services along with the information necessary to bind a third-party application as a consumer of that service. These services are described here: * [API metadata](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/1.0.1/api/govstack_im_service_metadata_api-0.3-swagger.json) and changes to the API definitions can be made by submitting a pull request on this repository. Additional APIs may be implemented by the Building Block, but the listed APIs define a minimal set of functionality that should be provided by any implementation of this Building Block.[Aleksander Reitsakas
Technical Reviewers: Tony Shannon, Saša Kovačević, Riham Moawad, Riham Fahmi, Aare Laponin, Manish Srivastava, Palab Saha, Surendra Singh Sucharia, Arvind Gupta, Gayatri. P., Shivank Singh Chauhan, Gavin Lyons
| External links updated | | 1.0 RC1 |Aleksander Reitsakas, Taylor Downs, Dr. P. S. Ramkumar, Aleksa Krolls
Reviewers:
Neil Roy, Aare Lapõnin, Fergal Marrinan, Surendrasingh Sucharia, Amy Darling
Software modules that can be deployed and combined in a standardized manner. Each Building Block is capable of working independently, but they can be combined to do much more. Building Blocks are composable, interoperable software modules that can be used across a variety of use cases. They are standards-based, preferably open-source, and designed for scale. Each Building Block exposes a set of services in the form of REST APIs that can be consumed by other Building Blocks or applications.
Ingress access is access from external applications to GovStack Building Blocks and applications.
Egress access is access from within GovStack Building Blocks and applications to external applications.
| | **Use Case** |A use case is a piece of functionality described as a sequence of actions (steps) to achieve a specific goal in a specific context of usage. E.g., in one use case, the Information Mediator Building Block may be used to let a Building Block access a service provided by another, or in a different use case may be used to relay an event notification from one Building Block to several other Building Blocks via a Publisher-Subscriber (Pub/Sub) model.
Each use case may involve a collection of modules or Building Blocks. A relatively small set of these Building Blocks can be readily applied to a wide variety of use cases in low-resource settings.
| | **API** | An application programming interface ([API](https://en.wikipedia.org/wiki/API)) is a connection between computers or between computer programs. It is a type of software interface, that offers a service to other pieces of software. A document or standard that describes how to build such a connection or interface is called an API specification. A computer system that meets this standard is said to implement or expose an API. The term API may refer either to the specification or to the implementation. | ## 3.5 Service Access | Term | Description | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Member** | A member is an organization that is authorized to communicate via the Information Mediator for a particular GovStack implementation. | | **Application** | An application is a running instance containing one or more Building Block instances and zero or more use case implementations. An application uses an Information Mediator Building Block to communicate with other Building Blocks or applications. An application typically has a single responsibility. | | **Service** | A service is a minimal piece of functionality provided by a Building Block or use case implementation. A service can be local (inside one application) or remote (between different applications). Remote services are consumed using REST protocol and they are described by OpenAPI specification. For example, the “registration” service might be accessed at a particular URL and allow a requester to “register a patient” by sending a POST request with proper patient data. | | **Security Server** | A security server is the main piece of software that is responsible for implementing the “service access” layer of the Information Mediator. This software acts as a gateway and is responsible for mediating requests between various members, applications, and services. It might be not just a single piece, but also a clustered or serverless deployment. | ## 3.6 Pub/Sub | Term | Description | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Publisher** | A Publisher produces events and sends them to rooms. Each event has an event type associated with it. Publishers can produce events of different types. | | **Room** | A Room is a Pub/Sub entity that handles the distribution of events. Each Room has a set of connected event types (E.g., the “birth” Room might contain three event types: “new\_birth”, “birth\_complication”, and “infant\_death”). | | **Subscriber** | A Subscriber can process events of a certain event type. Subscribers are independent of each other and their business logic is different (as rule). Each subscriber processes events from their own perspective. | # 4 Key Digital Functionalities Key Digital Functionalities describe the core (required) functions that this Building Block must be able to perform. The Information Mediator Building Block should provide the following functionalities, organized across 6 different components or layers ### 4.1 Service Access Service access is the facility through which different GovStack Building Blocks and applications may communicate securely with each other. The service access layer should provide an administrative interface that can be used to define users and roles and what access each should have. This layer should also include a registration interface which allows new members to onboard to the Information Mediator. The Service Access layer will also provide functionality to allow or restrict secure access to various resources, route requests to the correct providers, connect service requestors to providers, and enforce access privileges and security policies ### 4.2 Directory Services The Directory service allows applications to discover resources or functionalities that are made available by the system. An application may query the Directory service to discover organizations, services, and APIs that are available to them ### 4.3 Pub/Sub Service The Pub/Sub service is a facility through which applications may publish and subscribe to different events identified by unique message types, enabling more efficient and resilient communication and application design. It should provide broadcast/multicast capabilities to facilitate faster, more resilient application design. ### 4.4 Logging Services The Information Mediator will manage a message log which will provide comprehensive audit trails for all services requests and access. ### 4.5 Monitoring Services The Monitoring services will provide both operational and environmental reporting and dashboards, allowing users to discover the volume of requests to particular services or organizations as well as environmental reports on hardware/network usage. ### 4.6 Scaling/Throughput Services The Information Mediator should support scaling services such as load balancers and allow for the addition of additional servers to manage increased system use. ### Out-of-scope requirements Note that the Information Mediator is not responsible for manipulating the payloads sent to and from various applications—in a sense, it is both the postal service and the roads/bridges/train tracks—but it does not read the contents of your mail. The Information Mediator Building Blocking does not handle communication between human users and applications. The Information Mediator Building Blocking does not handle ingress and egress access from external applications to and from GovStack’s internal components (This may be handled by a public API Gateway in the domain of the Security Building Block). The current specification does not assume the possibility of different Information Mediator implementations working together. In one instance of the GovStack ecosystem, only one implementation of the Information Mediator Building Block can exist. The following requirements have been identified as “out of scope” because they fall into the realm of the Workflow Building Block or the Security Building Block. * Supports the handling of transactions and associated communication errors and exceptions. * Translates data from one format to another, and interoperates with handshake protocols to enable interoperability between different ICT (Information and communication technology) Building Blocks duplications. * Map data structures and fields from the identification system to the registration system and vice versa. * Hold authentication and credentials for each system. * Allow the definition of steps for a particular transaction. * Provide an API for both systems to access – and execute all necessary steps for that transaction (including error handling, retries, and notifications). * Provide an API for external systems to access GovStack Building Blocks. ### **Future Scope** In future iterations of this specification, we may take into consideration more broad API-management standards which include multiple domains, such as those proposed by the United Kingdom's Government in their [Defining an API management strategy](https://www.gov.uk/guidance/defining-an-api-management-strategy) and the section on [Gov.UK’s API Management Strategy Document](#4.3.3.2-gov.uks-api-management-strategy-document). # 5 Cross-Cutting Requirements This section will highlight important requirements or describe any additional cross-cutting requirements that apply to this Building Block. ## 5.1 Requirements The cross-cutting requirements described in this section are an extension of the cross-cutting requirements defined in the [Architecture Blueprint document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements) and the [Security Requirements](https://govstack.gitbook.io/specification/v/1.0/security-requirements). This section will describe any additional cross-cutting requirements for this Building Block or differences with the Architecture Blueprint. ## **5.1.1** GET and PUT APIs must be Idempotent (REQUIRED) Application APIs will contain POST endpoints which are not idempotent. GET/PUT/DELETE APIs are idempotent relative to Information Mediator Building Block, but the idempotents of intermediated services depend on the service provider and cannot be specified at this level. ## **5.1.2** Databases should not Include Business Logic (RECOMMENDED) Databases should not include business logic. We propose that this is a design recommendation which intends to make business logic all live in a clearly visible and accessible location. This may not always be followed, as the benefits of stored procedures (e.g., in reducing database transaction round-trips, etc.) may sometimes outweigh this general design recommendation. ## **5.1.3** Design for Asynchronous First (RECOMMENDED) The Service Access Layer of the Information Mediator is synchronous first. The Pub/Sub Layer is asynchronous. ## **5.1.4** Use Standardized Data Formats for Interchange (REQUIRED) Standard formats are used for communication with other Building Blocks. Inside Building Block non-standard protocols can be used. ## **5.1.5** Use Web Hooks for Callbacks (REQUIRED) OpenAPI spec 3.0 is supported in the first version. Moving to OpenAPI 3.1 is planned for the future. ## 5.2 Performance Requirements The Performance Requirements specify the basic parameters that an implementing government might use to establish performance requirements for scalability, throughput, and response times when reasonable/necessary. For example: * Minimum Throughput = 100 requests/sec. * Maximum Latency = 1 sec. * Concurrency = 1000 concurrent requests. * All solutions MUST be able to monitor and report, including but not limited to, resource consumption, throughput, latency, average latency, queue depth/backlogs, etc. * All of these indicators MUST be available through an administrative API. * Ideally, all Building Blocks should be able to run a “monitoring agent” which handles reporting out logs, requests, Building Block-specific indicators, etc. to a monitoring service (e.g. [Datadog](https://www.datadoghq.com/)) * The local monitoring agent should be configurable via web interface. * Retries and back-off strategies must be configurable. * Specific “[Scaling/Throughput](https://mediator.govstack.global/im-23q4/pages/AErmYUyYRmYjJFMF0Aon#6.6-scaling-throughput)” requirements are in the functional requirements section. ## 5.3 Standards The following standards are applicable to data structures in the Information Mediator Building Block: # 6 Functional Requirements This section lists the technical capabilities of this Building Block. The functional requirements section lists the technical capabilities that this Building Block should have. These requirements should be sufficient to deliver all functionality that is listed in the [Key Digital Functionalities](/im-23q4/4-key-digital-functionalities) section. The Information Mediator is used only for interaction between Applications/Building Blocks for the transfer of data at an API level only. Details in the [Architecture and Nonfunctional Requirements document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements/6-onboarding). These functional requirements do not define specific APIs, they provide a list of information about functionality that must be implemented within the Building Block. Detailed design and feature lists of these blocks can be customized by developers to optimally match specific target implementation needs. ## **6.1** Service Access #### **6.1.1 Administrative Interface** * There should be different types of administrative roles that provide different levels of access (RECOMMENDED) * There should be administrative roles on different levels of the ecosystem (RECOMMENDED): * global administrator role of the whole ecosystem, * local administrator role of Member, * an administrative role for configuring an Application. * There should be an ecosystem-level administrative interface that allows an Information Mediator administrator (with appropriate authentication) to register/deregister members and applications (RECOMMENDED) * There should be a security-server-level interface that allows an administrator for a single security server to manage members, applications, and services that live under a single security server (RECOMMENDED) * Open IAM (or another Identity and Access Management Solution) must be able to create/edit/delete admin users for the Information Mediator interface. [Security Building Block specification 1.0.1 section 6-1](https://govstack.gitbook.io/specification/v/1.0/security-requirements/6-security-building-block-modules) (REQUIRED) #### **6.1.2 Registration** (REQUIRED) * Registering a member/organization requires: * That the new member provides details for registration. * That the administrator verifies and accepts the request for registration. * Registering an application requires: * That the new member provides details for registration. * That the administrator verifies and accepts the request for registration. * Registering a service requires (in terms of API endpoints): * That the owner of the application provides: * OpenAPI 3.0 specification document (i.e., includes base paths like “/api/patients”, “/api/visits”, etc.). * Description. * List of enabled endpoints of OpenAPI specifications (the requirement is that we can enable/disable endpoints). * Managing a list of allowed consumers for services requires: * Application developers may access the directory service (described below) to select the services that they want to consume (OPTIONAL) * The request/approval/addition of allowed consumers process is a business-first process with manual steps (The hard part is negotiating data sharing agreements and signing contracts when that is completed the Information Mediator administrator can easily modify the allowed consumers' list manually) * An application must specify which member/application/service they want to access * The provider of that service must decide if the consumer is allowed to * Once approved, the requesting application will be added to the list of allowed applications for the requested service #### **6.1.3 Accessing Services** * To make a request to another service via the Information Mediator, an application must(REQUIRED): * Using REST, make a valid HTTPS request to the local Information Mediator security server with headers that identifies itself at the application level. * The components of the request must be (with color code): * Security server URL; * API version; * Instance (e.g., Country); * Domain of member; * Member (e.g., Ministry of X); * Application; * Service (OpenAPI file); * Path * Endpoint. * Query parameters. **Example Only**\ *Sample GET Request:*\ url-of-local-information-mediator-security-server/r1/INDIA/GOV/ministry-of-agriculture-karnataka/market-linkages-app/inventory-service/v1/check-level/apples?fresh=true\ *Response*: { data: 7 }\ \ *Sample POST Request:*\ url-of-local-information-mediator-security-server/r1/INDIA/PVT/tata-buyers-corp-karnataka/small-farmer-buyers-app/inventory-service/v1/supply/apples\ *With body*: {“qtyAvailable”: 4}\ *Response*: { result: “Stock level report created.” }\ \ *How to interpret the above request paths:*\ SECURITY-SERVER-URL/r1/INSTANCE/DOMAIN/MEMBER/APPLICATION/SERVICE/PATH * Note that all applications are making requests to the security server, which runs over the private network segment, rather than making requests to other applications directly over the public internet. (This is one of the main points of the security server and Information Mediator architecture.) ## 6.2 Directory Services * At development time, to see which resources are available on GovStack, the administrator/developer of application A may send requests to the security server to see(OPTIONAL): * organizations with descriptions; * services (either all services, or services that the requester is authorized to access); * an OpenAPI specification, detailing the endpoints and requirements for the service/API. * A view layer allowing for easy exploration of ALL clients, applications, and services should be provided. (Note that, “under the hood”, this layer may make use of the APIs described above or be implemented via a separate API.)(RECOMMENDED) ## 6.3 Pub/Sub Service #### 6.3.1 Defining Rooms and Event Types * The Pub/Sub layer must provide a mechanism for registering Rooms (REQUIRED) * The Pub/Sub layer must provide a mechanism for registering event types (REQUIRED) * For each event type, an OpenAPI description of a service with JSON schema for the required payload shape must be defined to create an event of the type (REQUIRED) * An interface for registering and viewing event types must be provided (REQUIRED) #### **6.3.2 Publishing a Message** * An application must be able to make a POST request with a valid JSON payload as the body and specify the message type to a Room provided by the Pub/Sub layer service (REQUIRED) #### 6.3.3 Delivering Messages * Published messages should be delivered to all subscribers (RECOMMENDED) * Published messages should be stored so that delivery may be done and retried asynchronously, e.g. if certain Subscribers are offline (RECOMMENDED) * Subscribers should be able to configure their retry strategies, overriding the default exponential backoff on retriable errors ([Google Pub/Sub example](https://cloud.google.com/pubsub/docs/handling-failures)) * If an active subscription exists but an error is received when forwarding a message to that service, the Pub/Sub layer should retry N times with a standard backoff. (An exponential backoff may be the default approach.) (RECOMMENDED) * The backoff and retry strategy should be configurable by an administrator with access to the Pub/Sub layer application * If a message cannot be delivered the Pub/Sub layer should drop that message (RECOMMENDED) #### 6.3.4 Managing Subscriptions * The Pub/Sub layer should provide an admin user interface to help create/manage subscriptions with the data below (RECOMMENDED) * The Pub/Sub layer should allow an administrator to view a list of active subscriptions (RECOMMENDED) * For registration via API, an application must be able to make a POST request to a service exposed by the Pub/Sub layer which defines which endpoints certain event types should be sent to (REQUIRED) #### 6.3.5 Event Logging and Audit Trail Generation * All events received and delivered must have a unique ID (REQUIRED) * All events received by the Pub/Sub layer must be logged or added to a log sync and those log entries must contain event metadata including the sender, timestamp, and event type, but may not include the event payload (REQUIRED) * All event delivery attempts must be logged or added to a log sync (REQUIRED) * For every event message received, the Information Mediator sends back an acknowledgment with the ID of the event to the respective Publisher (REQUIRED) * There must be a possibility to search and view the status of messages, for example, a message with type X to seven subscribers had been successfully delivered to all seven (REQUIRED) ## 6.4 Logging Services * The Information Mediator Building Block maintains a message log (REQUIRED) * The purpose of the message log is to provide means to prove the reception of a regular request or response message to a third party. Messages exchanged between Information Mediator Building Block are signed and encrypted. For every regular request and response, the security server produces a completely signed, and timestamped document. At a minimum, the log must store metadata that identifies a specific message, the status of transaction carried out on that message by Information Mediator Building Block, along with source ID and date time stamp. * The Information Mediator Building Block has full audit trail capabilities (REQUIRED) * The Information Mediator Building Block keeps an audit log. The audit log events are generated by the user interface when the administrator changes the system's state or configuration. The administrator's actions are logged regardless of whether the outcome was a success or a failure. The system must be capable of emitting statistical reports for a given organization, application, service, or consumer and status. ## 6.5 Monitoring Services * Operational monitoring (REQUIRED) * Operational monitoring provides details about the requested exchange, such as the ID-s of the client and the service, various attributes of the message read from the message header, request and response timestamps, sizes, etc., but not the actual payload of messages. * The operational monitoring daemon collects and shares operational monitoring data of the Information Mediator Building Block as part of request exchange, shares this data, and calculates and shares health statistics (the timestamps and the number of successful/unsuccessful requests, various metrics of the duration and message size of the requests, etc.). * The operational monitoring daemon makes operational and health data available to the owner of the security server, regular clients, and the central monitoring client via the security server. (For example, local health data may be made available for external monitoring systems.) * The owner of the security server and the central monitoring client are able to query the records of all clients. For a regular client, only the records associated with that client are available. * Environmental monitoring (REQUIRED) * Environmental monitoring provides details of the security servers such as operating system, memory, disk space, CPU load, traffic load, running processes, installed packages, etc. in a chosen date range. * Environmental monitoring provides a standard endpoint that can be accessed with a client (e.g. Java's console application if using Java Management Extensions). * It is possible to limit what allowed non-owners can request via environmental monitoring data requests. The security server owner will always get the full data set as requested. ## 6.6 Scaling/Throughput Services * The Information Mediator Building Block should support provider-side high availability setup if needed (RECOMMENDED) * Busy production systems may need a scalable performance in addition to high availability. The Information Mediator Building Block supports external load balancing mechanisms to address both of these problems simultaneously. A load balancer is added in front of a security server cluster to route the requests based on a selected algorithm (OPTIONAL) * The team in charge of deploying the security server application on their hardware must consider the network infrastructure including a load balancer, etc. The requirements for network infrastructure must be handled/defined in a (yet to exist) “Network Building Block”(REQUIRED) # 7 Data Structures This section provides information on the core data structures/data models that are used by this Building Block. The resource model shows the relationship between data objects that are used by the Information Mediator Building Block. The data elements provide detail for the resource model. All data element schemas can be viewed, commented on, and modified in the [schemas section of the Information Mediator Building Block GitHub repository](https://github.com/GovStackWorkingGroup/bb-information-mediator/tree/1.0-QA/schemas). ## 7.1 Resource Model The Mediator Building Block key element is Service. The Service is used by a consuming Building Block or an application and offered by a provider Building Block or Application. Both Provider and Consumer must be Members of the Mediator Building Block. Members of the Information Mediator Building Block can be an organization (governmental or not, business or not) or a person (citizen as a rule).  To become a Member of Mediator Building Block participants must fulfill declared requirements and apply for onboarding. In the process of onboarding a Member is registered with Mediator Building Block and gets credentials to connect to Mediator Building Block. Normally it is done the way that a Member provides a certificate of recognized Certification Authority (CA), and requests signed with this certificate are considered legitimate requests of the Member. A member entity can access the Pub/Sub configuration and register a room to publish its own event type as a publisher through its own admin. Members can browse a directory of Services available in the Mediator Building Block. Each service is described in OpenAPI. ## 7.2 Data Elements ### 7.2.1 Member **Fields:** | Name | Type | Description | | ------------ | ----------- | -------------------------------------------------------- | | class | enumerator | namespace for member\_code, e.g. GOV or COM | | code | string | unique id of member in registry defined by member\_class | | signing\_key | string | name of key used to sign on behalf of Member | | certificate | certificate | certificate of member issued by trusted CA | Schema reference [member.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/23Q4/schemas/member.json) ### 7.2.2 Application **Fields:** | Name | Type | Description | | ---------------- | ----------- | ------------------------------------------- | | code | string | unique name of application in member scope | | connection\_type | string | protocol used for connection: http or https | | certificate | certificate | TLS certificate | Schema reference [application.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/23Q4/schemas/application.json) ### 7.2.3 Service **Fields:** | Name | Type | Description | | ---------------- | ------ | ---------------------------------------------- | | code | string | unique name of OpenAPI in scope of application | | description\_url | url | location of OpenAPI service description | | service\_url | url | service provisioning network address | | ACL | object | description of access rights | Schema reference [service.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/23Q4/schemas/service.json) ## **7.3 PubSub Layer** ### 7.3.1 Resource Model The Resource Model is an extension of the Access Layer model:  ### 7.3.2 Data Elements #### 7.3.2.1 Event An event is a message—a set of data sent to a topic. Each event has an id. The event corresponds to the message. Data elements of an event are described by event type OpenAPI description. #### 7.3.2.2 Event Type An event type is schema definition for an event. Each event type is owned by a Room of a certain authority. (E.g. the Ministry of Health might own Room with the “new\_birth” event type and define its schema.) #### 7.3.2.3 Publisher A candidate application playing the role of **IM-Publisher** must be able to emit events to a specific Room. | Data Element | Default format | Description | | ------------ | -------------- | -------------------------------- | | id | string | application id of this publisher | | name | string | OPTIONAL | #### **7.3.2.4 Room** *(N.B., this is often called a "topic" and we may shift to that in later versions.)* A candidate application playing the role of **IM-Room** must handle the distribution of events. Each Room has a set of connected event types (e.g., the “birth” room might contain three event types: “new\_birth”, “birth\_complication”, and “infant\_death”). A room is located in the member’s local Information Mediator Building Block implementation and the member is responsible for all types of events in that particular room. | Data Element | Default format | Description | | ------------ | -------------- | --------------------------- | | id | string | application id of this room | | name | string | OPTIONAL | #### 7.3.2.5 Subscriber A candidate application playing the role of **IM-Subscriber** must be able to process events. It is done by defining a Service that will be called by Room to deliver an event. Subscribers are independent of each other and their business logic differs (as a rule). Each subscriber processes events from their own perspective. | Data Element | Default format | Description | | ------------ | -------------- | --------------------------------- | | id | string | application id of this subscriber | | name | string | OPTIONAL | #### 7.3.2.6 Subscription | Data Element | Default format | Description | | -------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | id | string | subscription id | | room\_id | string | room id | | subscriber\_id | string | subscriber id | | event\_type | string | A filter expression that allows a subscriber to subscribe to only certain message\_types that are published to the room they're subscribed to. | | mode | enum | delivery mode | | details | object | details of delivery, like time to live, repetition policy, etc. | # 8 Service APIs This section provides a reference for APIs that should be implemented by this Building Block. The [GovStack non-functional requirements document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements/6-onboarding) provides additional information on how 'adaptors' may be used to translate an existing API to the patterns described here. This section also provides guidance on how candidate products are tested and how GovStack validates a product's API against the API specifications defined here. The tests for the Information Mediator Building Block can be found in [this GitHub repository](https://github.com/GovStackWorkingGroup/bb-information-mediator/tree/main/test/openAPI). The majority of functions provided by the Information Mediator Building Block are either defined in the “service access flow” or configured by the administrator via the web User Interface. There is, however, a “Directory Service” which can provide listings of clients, methods, and available API specifications for services on the Information Mediator. The directory is managed by admins of members. The directory service centralizes and offers knowledge of all enrolled members and their services along with the information necessary to bind a third-party application as a consumer of that service. These services are described here: * [API metadata](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/23Q4/api/GovStack_IM_Directory_Services_API.yaml) and changes to the API definitions can be made by submitting a pull request on this repository. Additional APIs may be implemented by the Building Block, but the listed APIs define a minimal set of functionality that should be provided by any implementation of this Building Block. The services can be accessed via the following Service APIs: ## 8.1 Service Access The full API definition of all available services is the set of all available OpenAPI descriptions. One can take any of the available OpenAPI descriptions and call service according to that description. This call must be forwarded to IM local Security Server and path part of the called URL must begin with the address of service in the form `/r1/{instanceId}/{memberClass}/{member}/{application}/{service}/` followed by the service path with possible query parameters. The address of the service may be already listed in the OpenAPI description or must be added to the path if not provided by OpenAPI. ## 8.2 Directory Services #### 8.2.1 Member Discovery At development time, to see which organizations are available on GovStack, an administrator of application A sends a GET request to the security server: url-of-local-information-mediator-security-server/r1/listClients The response is an **array of organizations** with descriptions. API MAY implement paging of output. {% openapi src="Aleksander Reitsakas
Technical Reviewers: Tony Shannon, Saša Kovačević, Riham Moawad, Riham Fahmi, Aare Laponin, Manish Srivastava, Palab Saha, Surendra Singh Sucharia, Arvind Gupta, Gayatri. P., Shivank Singh Chauhan, Gavin Lyons
| External links updated | | 1.0.0 RC1 |Aleksander Reitsakas, Taylor Downs, Dr. P. S. Ramkumar, Aleksa Krolls
Reviewers:
Neil Roy, Aare Lapõnin, Fergal Marrinan, Surendrasingh Sucharia, Amy Darling
1.0.0
previously 23Q4
Software modules that can be deployed and combined in a standardized manner. Each Building Block is capable of working independently, but they can be combined to do much more. Building Blocks are composable, interoperable software modules that can be used across a variety of use cases. They are standards-based, preferably open-source, and designed for scale. Each Building Block exposes a set of services in the form of REST APIs that can be consumed by other Building Blocks or applications.
Ingress access is access from external applications to GovStack Building Blocks and applications.
Egress access is access from within GovStack Building Blocks and applications to external applications.
| | **Use Case** |A use case is a piece of functionality described as a sequence of actions (steps) to achieve a specific goal in a specific context of usage. E.g., in one use case, the Information Mediator Building Block may be used to let a Building Block access a service provided by another, or in a different use case may be used to relay an event notification from one Building Block to several other Building Blocks via a Publisher-Subscriber (Pub/Sub) model.
Each use case may involve a collection of modules or Building Blocks. A relatively small set of these Building Blocks can be readily applied to a wide variety of use cases in low-resource settings.
| | **API** | An application programming interface ([API](https://en.wikipedia.org/wiki/API)) is a connection between computers or between computer programs. It is a type of software interface, that offers a service to other pieces of software. A document or standard that describes how to build such a connection or interface is called an API specification. A computer system that meets this standard is said to implement or expose an API. The term API may refer either to the specification or to the implementation. | ## 3.5 Service Access | Term | Description | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Member** | A member is an organization that is authorized to communicate via the Information Mediator for a particular GovStack implementation. | | **Application** | An application is a running instance containing one or more Building Block instances and zero or more use case implementations. An application uses an Information Mediator Building Block to communicate with other Building Blocks or applications. An application typically has a single responsibility. | | **Service** | A service is a minimal piece of functionality provided by a Building Block or use case implementation. A service can be local (inside one application) or remote (between different applications). Remote services are consumed using REST protocol and they are described by OpenAPI specification. For example, the “registration” service might be accessed at a particular URL and allow a requester to “register a patient” by sending a POST request with proper patient data. | | **Security Server** | A security server is the main piece of software that is responsible for implementing the “service access” layer of the Information Mediator. This software acts as a gateway and is responsible for mediating requests between various members, applications, and services. It might be not just a single piece, but also a clustered or serverless deployment. | ## 3.6 Pub/Sub | Term | Description | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Publisher** | A Publisher produces events and sends them to rooms. Each event has an event type associated with it. Publishers can produce events of different types. | | **Room** | A Room is a Pub/Sub entity that handles the distribution of events. Each Room has a set of connected event types (E.g., the “birth” Room might contain three event types: “new\_birth”, “birth\_complication”, and “infant\_death”). | | **Subscriber** | A Subscriber can process events of a certain event type. Subscribers are independent of each other and their business logic is different (as rule). Each subscriber processes events from their own perspective. | # 4 Key Digital Functionalities Key Digital Functionalities describe the core (required) functions that this Building Block must be able to perform. The Information Mediator Building Block should provide the following functionalities, organized across 6 different components or layers ### 4.1 Service Access Service access is the facility through which different GovStack Building Blocks and applications may communicate securely with each other. The service access layer should provide an administrative interface that can be used to define users and roles and what access each should have. This layer should also include a registration interface which allows new members to onboard to the Information Mediator. The Service Access layer will also provide functionality to allow or restrict secure access to various resources, route requests to the correct providers, connect service requestors to providers, and enforce access privileges and security policies ### 4.2 Directory Services The Directory service allows applications to discover resources or functionalities that are made available by the system. An application may query the Directory service to discover organizations, services, and APIs that are available to them ### 4.3 Pub/Sub Service The Pub/Sub service is a facility through which applications may publish and subscribe to different events identified by unique message types, enabling more efficient and resilient communication and application design. It should provide broadcast/multicast capabilities to facilitate faster, more resilient application design. ### 4.4 Logging Services The Information Mediator will manage a message log which will provide comprehensive audit trails for all services requests and access. ### 4.5 Monitoring Services The Monitoring services will provide both operational and environmental reporting and dashboards, allowing users to discover the volume of requests to particular services or organizations as well as environmental reports on hardware/network usage. ### 4.6 Scaling/Throughput Services The Information Mediator should support scaling services such as load balancers and allow for the addition of additional servers to manage increased system use. ### Out-of-scope requirements Note that the Information Mediator is not responsible for manipulating the payloads sent to and from various applications—in a sense, it is both the postal service and the roads/bridges/train tracks—but it does not read the contents of your mail. The Information Mediator Building Blocking does not handle communication between human users and applications. The Information Mediator Building Blocking does not handle ingress and egress access from external applications to and from GovStack’s internal components (This may be handled by a public API Gateway in the domain of the Security Building Block). The current specification does not assume the possibility of different Information Mediator implementations working together. In one instance of the GovStack ecosystem, only one implementation of the Information Mediator Building Block can exist. The following requirements have been identified as “out of scope” because they fall into the realm of the Workflow Building Block or the Security Building Block. * Supports the handling of transactions and associated communication errors and exceptions. * Translates data from one format to another, and interoperates with handshake protocols to enable interoperability between different ICT (Information and communication technology) Building Blocks duplications. * Map data structures and fields from the identification system to the registration system and vice versa. * Hold authentication and credentials for each system. * Allow the definition of steps for a particular transaction. * Provide an API for both systems to access – and execute all necessary steps for that transaction (including error handling, retries, and notifications). * Provide an API for external systems to access GovStack Building Blocks. ### **Future Scope** In future iterations of this specification, we may take into consideration more broad API-management standards which include multiple domains, such as those proposed by the United Kingdom's Government in their [Defining an API management strategy](https://www.gov.uk/guidance/defining-an-api-management-strategy) and the section on [Gov.UK’s API Management Strategy Document](#4.3.3.2-gov.uks-api-management-strategy-document). # 5 Cross-Cutting Requirements This section will highlight important requirements or describe any additional cross-cutting requirements that apply to this Building Block. ## 5.1 Requirements The cross-cutting requirements described in this section are an extension of the cross-cutting requirements defined in the [Architecture Blueprint document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements) and the [Security Requirements](https://govstack.gitbook.io/specification/v/1.0/security-requirements). This section will describe any additional cross-cutting requirements for this Building Block or differences with the Architecture Blueprint. ### **5.1.1** GET and PUT APIs must be Idempotent (REQUIRED) Application APIs will contain POST endpoints which are not idempotent. GET/PUT/DELETE APIs are idempotent relative to Information Mediator Building Block, but the idempotents of intermediated services depend on the service provider and cannot be specified at this level. ### **5.1.2** Databases should not Include Business Logic (RECOMMENDED) Databases should not include business logic. We propose that this is a design recommendation which intends to make business logic all live in a clearly visible and accessible location. This may not always be followed, as the benefits of stored procedures (e.g., in reducing database transaction round-trips, etc.) may sometimes outweigh this general design recommendation. ### **5.1.3** Design for Asynchronous First (RECOMMENDED) The Service Access Layer of the Information Mediator is synchronous first. The Pub/Sub Layer is asynchronous. ### **5.1.4** Use Standardized Data Formats for Interchange (REQUIRED) Standard formats are used for communication with other Building Blocks. Inside Building Block non-standard protocols can be used. ### **5.1.5** Use Web Hooks for Callbacks (REQUIRED) OpenAPI spec 3.0 is supported in the first version. Moving to OpenAPI 3.1 is planned for the future. ## 5.2 Performance Requirements The Performance Requirements specify the basic parameters that an implementing government might use to establish performance requirements for scalability, throughput, and response times when reasonable/necessary. For example: * Minimum Throughput = 100 requests/sec. * Maximum Latency = 1 sec. * Concurrency = 1000 concurrent requests. * All solutions MUST be able to monitor and report, including but not limited to, resource consumption, throughput, latency, average latency, queue depth/backlogs, etc. * All of these indicators MUST be available through an administrative API. * Ideally, all Building Blocks should be able to run a “monitoring agent” which handles reporting out logs, requests, Building Block-specific indicators, etc. to a monitoring service (e.g. [Datadog](https://www.datadoghq.com/)) * The local monitoring agent should be configurable via web interface. * Retries and back-off strategies must be configurable. * Specific “[Scaling/Throughput](https://mediator.govstack.global/pages/LPb9h8GPp7kffWEcTJ2t#6.6-scaling-throughput)” requirements are in the functional requirements section. ## 5.3 Standards The following standards are applicable to data structures in the Information Mediator Building Block: # 6 Functional Requirements This section lists the technical capabilities of this Building Block. The functional requirements section lists the technical capabilities that this Building Block should have. These requirements should be sufficient to deliver all functionality that is listed in the [Key Digital Functionalities](/4-key-digital-functionalities) section. The Information Mediator is used only for interaction between Applications/Building Blocks for the transfer of data at an API level only. Details in the [Architecture and Nonfunctional Requirements document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements/6-onboarding). These functional requirements do not define specific APIs, they provide a list of information about functionality that must be implemented within the Building Block. Detailed design and feature lists of these blocks can be customized by developers to optimally match specific target implementation needs. ## **6.1 Administrative Interface** * There should be different types of administrative roles that provide different levels of access (RECOMMENDED) * There should be administrative roles on different levels of the ecosystem (RECOMMENDED): * global administrator role of the whole ecosystem, * local administrator role of Member, * an administrative role for configuring an Application. * There should be an ecosystem-level administrative interface that allows an Information Mediator administrator (with appropriate authentication) to register/deregister members and applications (RECOMMENDED) * There should be a security-server-level interface that allows an administrator for a single security server to manage members, applications, and services that live under a single security server (RECOMMENDED) * Open IAM (or another Identity and Access Management Solution) must be able to create/edit/delete admin users for the Information Mediator interface. [Security Building Block specification 1.0.1 section 6-1](https://govstack.gitbook.io/specification/v/1.0/security-requirements/6-security-building-block-modules) (REQUIRED) ## **6.2 Registration** Services * Provide a mechanism for registering a member/organization, where the new member provides details for registration and the administrator verifies and accepts the request for registration (REQUIRED) * Provide a mechanism for registering an application where the new member provides details for registration and the administrator verifies and accepts the request for registration (REQUIRED) * Provide a mechanism for registering a service (API endpoints), which includes an OpenAPI 3.0 specification document and description, along with the ability to enable and disable endpoints (REQUIRED) * Provide a mechanism for application developers to access the directory service (described below) to select the services that they want to consume (RECOMMENDED) * The request/approval/addition of allowed consumers process is a business-first process with manual steps (The hard part is negotiating data sharing agreements and signing contracts when that is completed the Information Mediator administrator can easily modify the allowed consumers' list manually) * An application must specify which member/application/service they want to access * The provider of that service must decide if the consumer is allowed to access the service. * Once approved, the requesting application will be added to the list of allowed applications for the requested service ## **6.3 Accessing Services** * The Information Mediator must allow a service to make a request to another service using REST calls (REQUIRED) * The REST request must use HTTPS/TLS to the local Information Mediator security server with headers that identifies itself at the application level. * The components of the request must be (with color code): * Security server URL; * API version; * Instance (e.g., Country); * Domain of member; * Member (e.g., Ministry of X); * Application; * Service (OpenAPI file); * Path * Endpoint. * Query parameters. **Example Only**\ *Sample GET Request:*\ url-of-local-information-mediator-security-server/r1/INDIA/GOV/ministry-of-agriculture-karnataka/market-linkages-app/inventory-service/v1/check-level/apples?fresh=true\ *Response*: { data: 7 }\ \ *Sample POST Request:*\ url-of-local-information-mediator-security-server/r1/INDIA/PVT/tata-buyers-corp-karnataka/small-farmer-buyers-app/inventory-service/v1/supply/apples\ *With body*: {“qtyAvailable”: 4}\ *Response*: { result: “Stock level report created.” }\ \ *How to interpret the above request paths:*\ SECURITY-SERVER-URL/r1/INSTANCE/DOMAIN/MEMBER/APPLICATION/SERVICE/PATH * Note that all applications are making requests to the security server, which runs over the private network segment, rather than making requests to other applications directly over the public internet. This is the desired situation and is one of the main principles of the Information Mediator Building Block architecture and the architecture of the Security Server ## 6.4 Directory Services * At development time, to see which resources are available on GovStack, the administrator/developer of application A may send requests to the security server to see organizations, services, and OpenAPI specifications. (RECOMMENDED) * A view layer allowing for easy exploration of ALL clients, applications, and services should be provided. (Note that, “under the hood”, this layer may make use of the APIs described above or be implemented via a separate API). (RECOMMENDED) ## 6.5 Pub/Sub Service * The Pub/Sub layer must provide a mechanism for registering Rooms (RECOMMENDED) * The Pub/Sub layer must provide a mechanism for registering event types (RECOMMENDED) * For each event type, an OpenAPI description of a service with JSON schema for the required payload shape must be defined to create an event of the type (RECOMMENDED) * An interface for registering and viewing event types must be provided (RECOMMENDED) * An application must be able to publish a message by making a POST request with a valid JSON payload as the body and specify the message type to a Room provided by the Pub/Sub layer service (RECOMMENDED) * Published messages should be delivered to all subscribers (RECOMMENDED) * Published messages should be stored so that delivery may be done and retried asynchronously, e.g. if certain Subscribers are offline (RECOMMENDED) * Subscribers should be able to configure their retry strategies, overriding the default exponential backoff on retriable errors ([Google Pub/Sub example](https://cloud.google.com/pubsub/docs/handling-failures)) * If an active subscription exists but an error is received when forwarding a message to that service, the Pub/Sub layer should retry N times with a standard backoff. (An exponential backoff may be the default approach.) (RECOMMENDED) * The backoff and retry strategy should be configurable by an administrator with access to the Pub/Sub layer application * If a message cannot be delivered the Pub/Sub layer should drop that message (RECOMMENDED) * The Pub/Sub layer should provide an admin user interface to help create/manage subscriptions with the data below (RECOMMENDED) * The Pub/Sub layer should allow an administrator to view a list of active subscriptions (RECOMMENDED) * For registration via API, an application must be able to make a POST request to a service exposed by the Pub/Sub layer which defines which endpoints certain event types should be sent to (RECOMMENDED) * All events received and delivered must have a unique ID (RECOMMENDED) * All events received by the Pub/Sub layer must be logged or added to a log sync and those log entries must contain event metadata including the sender, timestamp, and event type, but may not include the event payload (RECOMMENDED) * All event delivery attempts must be logged or added to a log sync (RECOMMENDED) * For every event message received, the Information Mediator sends back an acknowledgment with the ID of the event to the respective Publisher (RECOMMENDED) * There must be a possibility to search and view the status of messages, for example, a message with type X to seven subscribers had been successfully delivered to all seven (RECOMMENDED) ## 6.5 Logging Services * The Information Mediator Building Block maintains a message log (REQUIRED) * The purpose of the message log is to provide means to prove the reception of a regular request or response message to a third party. Messages exchanged between Information Mediator Building Block are signed and encrypted. For every regular request and response, the security server produces a completely signed, and timestamped document. At a minimum, the log must store metadata that identifies a specific message, the status of transaction carried out on that message by Information Mediator Building Block, along with source ID and date time stamp. * The Information Mediator Building Block must have full audit trail capabilities (REQUIRED) * The Information Mediator Building Block keeps an audit log. The audit log events are generated by the user interface when the administrator changes the system's state or configuration. The administrator's actions are logged regardless of whether the outcome was a success or a failure. The system must be capable of emitting statistical reports for a given organization, application, service, or consumer and status. ## 6.6 Monitoring Services * The Information mediator must provide operational monitoring including information about requests and service health, (REQUIRED) * Operational monitoring provides details about the requested exchange, such as the ID-s of the client and the service, various attributes of the message read from the message header, request and response timestamps, sizes, etc., but not the actual payload of messages. * The operational monitoring daemon collects and shares operational monitoring data of the Information Mediator Building Block as part of request exchange, shares this data, and calculates and shares health statistics such as the timestamps and the number of successful/unsuccessful requests, various metrics of the duration and message size of the requests, etc. * The operational monitoring daemon makes operational and health data available to the owner of the security server, regular clients, and the central monitoring client via the security server. For example, local health data may be made available for external monitoring systems. * The owner of the security server and the central monitoring client are able to query the records of all clients. For a regular client, only the records associated with that client are available. * The Information mediator must provide environmental monitoring services such as CPU load, disk space, and traffic (REQUIRED) * Environmental monitoring provides details of the security servers such as operating system, memory, disk space, CPU load, traffic load, running processes, installed packages, etc. in a chosen date range. * Environmental monitoring provides a standard endpoint that can be accessed with a client (e.g. Java's console application if using Java Management Extensions). * It is possible to limit what allowed non-owners can request via environmental monitoring data requests. The security server owner will always get the full data set as requested. ## 6.7 Scaling/Throughput Services * The Information Mediator Building Block should support provider-side high availability setup if needed (RECOMMENDED) * Busy production systems may need a scalable performance in addition to high availability. The Information Mediator Building Block supports external load balancing mechanisms to address both of these problems simultaneously. A load balancer is added in front of a security server cluster to route the requests based on a selected algorithm. (RECOMMENDED) * The team in charge of deploying the security server application on their hardware must consider the network infrastructure including a load balancer, etc. The requirements for network infrastructure are to be described in the Information Mediator Building Block implementation guide. (REQUIRED) # 7 Data Structures This section provides information on the core data structures/data models that are used by this Building Block. The resource model shows the relationship between data objects that are used by the Information Mediator Building Block. The data elements provide detail for the resource model. All data element schemas can be viewed, commented on, and modified in the [schemas section of the Information Mediator Building Block GitHub repository](https://github.com/GovStackWorkingGroup/bb-information-mediator/tree/1.0-QA/schemas). ## 7.1 Resource Model The Mediator Building Block key element is Service. The Service is used by a consuming Building Block or an application and offered by a provider Building Block or Application. Both Provider and Consumer must be Members of the Mediator Building Block. Members of the Information Mediator Building Block can be an organization (governmental or not, business or not) or a person (citizen as a rule).  To become a Member of Mediator Building Block participants must fulfill declared requirements and apply for onboarding. In the process of onboarding a Member is registered with Mediator Building Block and gets credentials to connect to Mediator Building Block. Normally it is done the way that a Member provides a certificate of recognized Certification Authority (CA), and requests signed with this certificate are considered legitimate requests of the Member. A member entity can access the Pub/Sub configuration and register a room to publish its own event type as a publisher through its own admin. Members can browse a directory of Services available in the Mediator Building Block. Each service is described in OpenAPI. ## 7.2 Data Elements ### 7.2.1 Member **Fields:** | Name | Type | Description | | ------------ | ----------- | -------------------------------------------------------- | | class | enumerator | namespace for member\_code, e.g. GOV or COM | | code | string | unique id of member in registry defined by member\_class | | signing\_key | string | name of key used to sign on behalf of Member | | certificate | certificate | certificate of member issued by trusted CA | Schema reference [member.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/main/schemas/member.json) ### 7.2.2 Application **Fields:** | Name | Type | Description | | ---------------- | ----------- | ------------------------------------------- | | code | string | unique name of application in member scope | | connection\_type | string | protocol used for connection: http or https | | certificate | certificate | TLS certificate | Schema reference [application.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/main/schemas/application.json) ### 7.2.3 Service **Fields:** | Name | Type | Description | | ---------------- | ------ | ---------------------------------------------- | | code | string | unique name of OpenAPI in scope of application | | description\_url | url | location of OpenAPI service description | | service\_url | url | service provisioning network address | | ACL | object | description of access rights | Schema reference [service.json](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/main/schemas/service.json) ## **7.3 PubSub Layer** ### 7.3.1 Resource Model The Resource Model is an extension of the Access Layer model:  ### 7.3.2 Data Elements #### 7.3.2.1 Event An event is a message—a set of data sent to a topic. Each event has an id. The event corresponds to the message. Data elements of an event are described by event type OpenAPI description. #### 7.3.2.2 Event Type An event type is schema definition for an event. Each event type is owned by a Room of a certain authority. (E.g. the Ministry of Health might own Room with the “new\_birth” event type and define its schema.) #### 7.3.2.3 Publisher A candidate application playing the role of **IM-Publisher** must be able to emit events to a specific Room. | Data Element | Default format | Description | | ------------ | -------------- | -------------------------------- | | id | string | application id of this publisher | | name | string | OPTIONAL | #### **7.3.2.4 Room** *(N.B., this is often called a "topic" and we may shift to that in later versions.)* A candidate application playing the role of **IM-Room** must handle the distribution of events. Each Room has a set of connected event types (e.g., the “birth” room might contain three event types: “new\_birth”, “birth\_complication”, and “infant\_death”). A room is located in the member’s local Information Mediator Building Block implementation and the member is responsible for all types of events in that particular room. | Data Element | Default format | Description | | ------------ | -------------- | --------------------------- | | id | string | application id of this room | | name | string | OPTIONAL | #### 7.3.2.5 Subscriber A candidate application playing the role of **IM-Subscriber** must be able to process events. It is done by defining a Service that will be called by Room to deliver an event. Subscribers are independent of each other and their business logic differs (as a rule). Each subscriber processes events from their own perspective. | Data Element | Default format | Description | | ------------ | -------------- | --------------------------------- | | id | string | application id of this subscriber | | name | string | OPTIONAL | #### 7.3.2.6 Subscription | Data Element | Default format | Description | | -------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | id | string | subscription id | | room\_id | string | room id | | subscriber\_id | string | subscriber id | | event\_type | string | A filter expression that allows a subscriber to subscribe to only certain message\_types that are published to the room they're subscribed to. | | mode | enum | delivery mode | | details | object | details of delivery, like time to live, repetition policy, etc. | # 8 Service APIs This section provides a reference for APIs that should be implemented by this Building Block. The [GovStack non-functional requirements document](https://govstack.gitbook.io/specification/v/1.0/architecture-and-nonfunctional-requirements/6-onboarding) provides additional information on how 'adaptors' may be used to translate an existing API to the patterns described here. This section also provides guidance on how candidate products are tested and how GovStack validates a product's API against the API specifications defined here. The tests for the Information Mediator Building Block can be found in [this GitHub repository](https://github.com/GovStackWorkingGroup/bb-information-mediator/tree/main/test/openAPI). The majority of functions provided by the Information Mediator Building Block are either defined in the “service access flow” or configured by the administrator via the web User Interface. There is, however, a “Directory Service” which can provide listings of clients, methods, and available API specifications for services on the Information Mediator. The directory is managed by admins of members. The directory service centralizes and offers knowledge of all enrolled members and their services along with the information necessary to bind a third-party application as a consumer of that service. These services are described here: * [API metadata](https://github.com/GovStackWorkingGroup/bb-information-mediator/blob/main/api/GovStack_IM_Directory_Services_API.yaml) and changes to the API definitions can be made by submitting a pull request on this repository. Additional APIs may be implemented by the Building Block, but the listed APIs define a minimal set of functionality that should be provided by any implementation of this Building Block. The services can be accessed via the following Service APIs: ## 8.1 Service Access The full API definition of all available services is the set of all available OpenAPI descriptions. One can take any of the available OpenAPI descriptions and call service according to that description. This call must be forwarded to IM local Security Server and path part of the called URL must begin with the address of service in the form `/r1/{instanceId}/{memberClass}/{member}/{application}/{service}/` followed by the service path with possible query parameters. The address of the service may be already listed in the OpenAPI description or must be added to the path if not provided by OpenAPI. ## 8.2 Directory Services ### 8.2.1 Member Discovery At development time, to see which organizations are available on GovStack, an administrator of application A sends a GET request to the security server: url-of-local-information-mediator-security-server/r1/listClients The response is an **array of organizations** with descriptions. API MAY implement paging of output. ## List clients defined in the instance > Clients of the security server have the capability to obtain a list\ > of potential service providers within a GovStack instance, including\ > both members and applications.\ > To do so, they should initiate an HTTP GET request to the security\ > server.\ > The specific request URL will be either \`This API takes provided filters from query parameters and returns a subset of access policies configured on the IM building block, based on the filters.\
\The result is paged, with page size based on the number of service and allowed application pairs. For example, when pageSize = 10 and the IM BB defines three services: A, B and C, each of which has 8 clients, the result will be split over 3 pages:\
\
Page 1: Service A, with all of its 8 allowed applications, Service B with 2 allowed applications.\
Page 2: Service B, with 6 allowed applications left over, Service C with 4 applications.\
Page 3: Service C, with 4 applications.\
\
Implementations can define additional query parameters for result display (such as ordering) and how next page tokens are implemented. It is expected that the next page token is not readable by API users and will be provided verbatim when querying the next page of results. \
```json {"openapi":"3.0.0","info":{"title":"GovStack IM Configuration Management API","version":"1.0.1"},"tags":[{"name":"rights","description":"Access rights management"}],"security":[{"auth":["read:rights"]}],"components":{"securitySchemes":{"auth":{"type":"oauth2","flows":{"authorizationCode":{"authorizationUrl":"https://example.com/oauth/authorize","tokenUrl":"https://example.com/oauth/token","scopes":{"read:conf":"Grants read access to IM configuration","create:conf":"Grants permission to create IM configuration","write:conf":"Grants write access to IM configuration","read:rights":"Grants permission to see what services are allowed","write:rights":"Grants permission to allow/deny IM services"}}}}},"schemas":{"PaginatedResult":{"type":"object","properties":{"nextPageToken":{"type":"string"}}},"ServiceAccessRights":{"type":"array","items":{"type":"object","required":["memberClass","memberCode","applicationId","serviceId","rights"],"properties":{"memberClass":{"description":"Kind of organisation. Namespace for organisation {code}","type":"string"},"memberCode":{"description":"Registration number (or identifier) of the organisation in {memberClass} namespace","type":"string"},"applicationId":{"type":"string","description":"Name of application"},"serviceId":{"description":"Name of service/API","type":"string"},"rights":{"description":"Applications that are allowed or denied access","type":"array","items":{"$ref":"#/components/schemas/AllowedApplication"}}}}},"AllowedApplication":{"description":"An identified member application that has the permission to access a service.","type":"object","required":["member","application"],"properties":{"memberClass":{"description":"Member or member group identifier to whom access is granted","type":"string"},"memberCode":{"description":"Member or member group identifier to whom access is granted","type":"string"},"applicationId":{"description":"Application of the member that to whom access is granted","type":"string"}}}}},"paths":{"/api/v1/rights/allow":{"get":{"tags":["rights"],"summary":"Retrieve configured access permissions (allow list)","description":"This API takes provided filters from query parameters and returns a subset of access policies configured on the IM building block, based on the filters.
The result is paged, with page size based on the number of service and allowed application pairs. For example, when pageSize = 10 and the IM BB defines three services: A, B and C, each of which has 8 clients, the result will be split over 3 pages:
Page 1: Service A, with all of its 8 allowed applications, Service B with 2 allowed applications.
Page 2: Service B, with 6 allowed applications left over, Service C with 4 applications.
Page 3: Service C, with 4 applications.
Implementations can define additional query parameters for result display (such as ordering) and how next page tokens are implemented. It is expected that the next page token is not readable by API users and will be provided verbatim when querying the next page of results.
","operationId":"getAllowed","parameters":[{"in":"query","name":"memberClass","description":"Filter by member class","schema":{"type":"string"},"required":false},{"in":"query","name":"memberCode","schema":{"type":"string"},"description":"Filter by member code","required":false},{"in":"query","name":"applicationId","schema":{"type":"string"},"description":"Filter by application ID","required":false},{"in":"query","name":"serviceId","schema":{"type":"string"},"description":"Filter by service ID","required":false},{"in":"query","name":"pageSize","description":"Number of access rights returned on one page","schema":{"type":"number","default":100},"required":false},{"in":"query","name":"nextPageToken","description":"Handle for the next page, if the result spans multiple pages. If not specified, there are no more results.","schema":{"type":"string"},"required":false}],"responses":{"200":{"description":"OK","content":{"application/json":{"schema":{"allOf":[{"$ref":"#/components/schemas/PaginatedResult"},{"type":"object","properties":{"allowList":{"$ref":"#/components/schemas/ServiceAccessRights"}}}]}}}},"401":{"description":"Caller does not have permissions to view access rights"}}}}}} ``` ## PATCH /api/v1/rights/allow > Allow access to services ```json {"openapi":"3.0.0","info":{"title":"GovStack IM Configuration Management API","version":"1.0.1"},"tags":[{"name":"rights","description":"Access rights management"}],"security":[{"auth":["write:rights"]}],"components":{"securitySchemes":{"auth":{"type":"oauth2","flows":{"authorizationCode":{"authorizationUrl":"https://example.com/oauth/authorize","tokenUrl":"https://example.com/oauth/token","scopes":{"read:conf":"Grants read access to IM configuration","create:conf":"Grants permission to create IM configuration","write:conf":"Grants write access to IM configuration","read:rights":"Grants permission to see what services are allowed","write:rights":"Grants permission to allow/deny IM services"}}}}},"schemas":{"ServiceAccessRights":{"type":"array","items":{"type":"object","required":["memberClass","memberCode","applicationId","serviceId","rights"],"properties":{"memberClass":{"description":"Kind of organisation. Namespace for organisation {code}","type":"string"},"memberCode":{"description":"Registration number (or identifier) of the organisation in {memberClass} namespace","type":"string"},"applicationId":{"type":"string","description":"Name of application"},"serviceId":{"description":"Name of service/API","type":"string"},"rights":{"description":"Applications that are allowed or denied access","type":"array","items":{"$ref":"#/components/schemas/AllowedApplication"}}}}},"AllowedApplication":{"description":"An identified member application that has the permission to access a service.","type":"object","required":["member","application"],"properties":{"memberClass":{"description":"Member or member group identifier to whom access is granted","type":"string"},"memberCode":{"description":"Member or member group identifier to whom access is granted","type":"string"},"applicationId":{"description":"Application of the member that to whom access is granted","type":"string"}}}}},"paths":{"/api/v1/rights/allow":{"patch":{"tags":["rights"],"summary":"Allow access to services","operationId":"setAllowed","requestBody":{"description":"","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ServiceAccessRights"}}},"required":false},"responses":{"200":{"description":"OK"},"401":{"description":"Caller does not have the necessary permissions to update access rights"}}}}}} ``` ## PATCH /api/v1/rights/deny > Deny access to services ```json {"openapi":"3.0.0","info":{"title":"GovStack IM Configuration Management API","version":"1.0.1"},"tags":[{"name":"rights","description":"Access rights management"}],"security":[{"auth":["write:rights"]}],"components":{"securitySchemes":{"auth":{"type":"oauth2","flows":{"authorizationCode":{"authorizationUrl":"https://example.com/oauth/authorize","tokenUrl":"https://example.com/oauth/token","scopes":{"read:conf":"Grants read access to IM configuration","create:conf":"Grants permission to create IM configuration","write:conf":"Grants write access to IM configuration","read:rights":"Grants permission to see what services are allowed","write:rights":"Grants permission to allow/deny IM services"}}}}},"schemas":{"ServiceAccessRights":{"type":"array","items":{"type":"object","required":["memberClass","memberCode","applicationId","serviceId","rights"],"properties":{"memberClass":{"description":"Kind of organisation. Namespace for organisation {code}","type":"string"},"memberCode":{"description":"Registration number (or identifier) of the organisation in {memberClass} namespace","type":"string"},"applicationId":{"type":"string","description":"Name of application"},"serviceId":{"description":"Name of service/API","type":"string"},"rights":{"description":"Applications that are allowed or denied access","type":"array","items":{"$ref":"#/components/schemas/AllowedApplication"}}}}},"AllowedApplication":{"description":"An identified member application that has the permission to access a service.","type":"object","required":["member","application"],"properties":{"memberClass":{"description":"Member or member group identifier to whom access is granted","type":"string"},"memberCode":{"description":"Member or member group identifier to whom access is granted","type":"string"},"applicationId":{"description":"Application of the member that to whom access is granted","type":"string"}}}}},"paths":{"/api/v1/rights/deny":{"patch":{"tags":["rights"],"summary":"Deny access to services","operationId":"setDenied","requestBody":{"description":"","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ServiceAccessRights"}}},"required":false},"responses":{"200":{"description":"OK"},"401":{"description":"Caller does not have the necessary permissions to update access rights"}}}}}} ``` # 9 Internal Workflows This section provides a detailed view of how this Building Block will interact with other Building Blocks to support common use cases. A workflow provides a detailed view of how the Information Mediator Building Block will interact with other building blocks to support common use cases. This section lists workflows that this building block must support. Other workflows may be implemented in addition to those listed. ## 9.1 Service Access Example Sequence diagram for accessing service {% @mermaid/diagram content="sequenceDiagram participant Requestor participant SS1 as IM Security Server of Requestor participant CS as IM Central Server participant CA as CA participant TSA as TSA participant SS2 as IM Security Server of Responder participant Responder Requestor ->> SS1 : Message X SS1 ->> CS : read configuration\* CS -->> SS1 : GovStack configuration SS1 ->> SS1 : find Responder SS1 ->> SS2 : Initiate TLS connection\* SS2 ->> SS1 : Responder cert SS1 ->> CA : OCSP check Responder cert\* CA -->> SS1 : yes/no SS1 ->> SS1 : compose digital document containing "Message X" SS1 ->> TSA : stamp document TSA -->> SS1: timestamp SS1 ->> SS1 : sign document SS1 ->> SS2 : send document over TLS SS2 ->> CA : OCSP check Requestor cert\* CA -->> SS2 : yes/no SS2 ->> SS2 : check document signature SS2 ->> SS2 : write message to log SS2 ->> SS2 : check access rights SS2 ->> Responder : Message X" %} The results of steps marked with '\*' can be cached and reused between calls. In this diagram: 1. Requestor makes a request to the local Information Mediator security server. 2. Local Infirmation Mediator security server finds in the configuration where to send request. 3. The Information Mediator signs & sends the request from application A to the security server for application B. 4. The security server for application B receives the request, validates the signature, and then forwards it to the application/service/endpoint. 5. The application/service/endpoint responds to request. 6. The security server for application B signs the response and sends it back to the security server for application A. 7. The security server for the application A validates the signature and forwards the response to application A. 8. This is all synchronous. Application A’s request is open/unresponded until step 6. ## 9.2 Pub/Sub ### 9.2.1 Registering a Room {% @mermaid/diagram content="sequenceDiagram participant Owner participant SS as IM Security Server participant CS as IM Central Server participant adm as Administrator Owner ->> SS: create Room SS ->> CS: register Room as application CS ->> CS: store request CS -->> SS: registration accepted adm ->> CS: confirm application creation request CS ->> CS: change configuration SS ->> CS: read configuration CS -->> SS: GovStack configuration " %} Room for Pub/Sub must be created before events can be described and processed. Room is created by its owner organization (Member), the organization responsible for the operation of events of a certain type. Creation request should be confirmed by GovStack administrator. ### 9.2.2 Defining event type {% @mermaid/diagram content="sequenceDiagram participant Owner participant SS as IM Security Server participant Server Owner ->> SS: define event type SS ->> Server: get OpenAPI description Server -->> SS: OpenAPI SS ->> SS: create service SS -->> Owner: done" %} Event type corresponds to a service description of a service accepting events for publishing and must be registered before usage. A single room might host multiple event types. E.g. in India, "The Ministry Of Home Affairs" can be the "Owner" of a Room where events of type "emergency" are published by "citizens" or "entities" and entities such as "ambulance services", "Fire-fighting services", Hospitals, etc., can be Subscribers to this Room. In this scenario, an event of type "Fire emergency" can be published by any citizen enrolled in this Room. The Ministry can choose to have one "emergency" Room in each town and enroll Subscribers relevant to respective regions. ### 9.2.3 Subscribing to events of a certain type {% @mermaid/diagram content="sequenceDiagram participant Subscriber participant SS1 as IM Security Server of Subscriber participant SS2 as IM Security Server of Room participant Room opt event type discovery (design time) Subscriber ->> SS1: request list of Members and Rooms SS1 -->> Subscriber: list of Members and Rooms Subscriber ->> Subscriber: chose Room Subscriber ->> SS1: request list of available event types (Services) SS1 ->> SS2: request list of available event types (Services) SS2 -->> SS1: list of event types SS1 -->> Subscriber: list of event types Subscriber ->> SS1: GET description of the event type SS1 ->> SS2: GET description of the event type SS2 -->> SS1: OpenAPI of the event type SS1 -->> Subscriber: OpenAPI of the event type Subscriber ->> SS1: register endpoint for receiving events SS1 -->> Subscriber: done end Subscriber ->> SS1: subscribe to event type SS1 ->> SS2: subscribe to event type SS2 ->> Room: subscribe Room ->> Room: register subscription Room -->> SS2: done SS2 --> SS1: done SS1 --> Subscriber: done" %} 1. The Subscriber requests that events of certain types be delivered to them. 2. The Subscriber specifies the desired delivery mode (push/pull). The Room and Subscriber conclude the delivery contract. Note that: * push delivery mode is when the Room sends events to the Subscriber’s API. In this case, Subscriber may request some QoS/SLA parameters of delivery. * pull delivery mode (OPTIONAL) means that the Room keeps a queue of events and the Subscriber can check that queue. 3. At a minimum, this Building Block MUST provide for a push delivery mode. It may also provide a pull mode based on convenience. Three endpoints to be declared per event type Pub/Sub instance: * an endpoint URL to be registered by every Subscriber on a per-event type basis. * (OPTIONAL) an endpoint URL on the Pub/Sub where all pull requests come from various Subscribers. * an endpoint in the Room to send messages for publication. ### 9.2.4 Event delivery {% @mermaid/diagram content="sequenceDiagram participant Publisher participant Room participant Subscriber Publisher ->> Room: POST event Room ->> Room: store event Room -->> Publisher: event id loop for all Subscribers alt PUSH delivery mode Room ->> Subscriber: POST event Subscriber -->> Room: done else PULL delivery mode Room ->> Room: Store event in Subscriber queue opt asynchronous activity Subscriber ->> Room: GET event of type Room -->> Subscriber: event Subscriber ->> Room: Acknowledge event end end end opt get details Subscriber -->> Publisher: request event details Publisher -->> Subscriber: details end " %} 1. The Publisher and the Room have a data access contract to establish trust for accessing information. * This is akin to the standard trust contract in the Information Mediator, the consumer must have rights to access a certain API. * Publishers have to sign a contract with the Room owner’s consent to gain access to specific Rooms and events. In the contract, Publishers declare what type of message will be published in a given Room. 2. The Publisher generates an event: * The Publisher makes a POST call to the Room service of a particular event type. * The Room stores the event and replies with the event id. Event id is generated by Room or is taken from the original event dataset if provided by the publisher. * The Room can implement a first in, first out (FIFO) event distribution policy. If the Room provides FIFO guarantees, the Room may: * (a) assign a key to the event. If a key attribute is defined in the schema of the event, the value of that attribute will be used. Otherwise, the Room can assign a key at its own discretion. * (b) assign a sequence number to the event. The sequence number must be monotonically increasing for events that share the same key. * The Room sends an immediate acknowledgment to the Publisher. 3. The Room distributes an event asynchronously as follows: * A reference to publisher and event id is added to the event dataset. * For each Subscriber: * (alt) If the mode is ‘push’, make a POST call to Subscriber GovStack service of the event type; * (alt) If the mode is ‘pull’, enqueue an event for request from the Subscriber; * There is a queue of events waiting to be processed per the Subscriber, such that the Subscriber might periodically check to see events waiting in their own queue, process those events, and clear the queue. * A pull mechanism is essential for resilience to network dropouts and traffic load balance at servers and for differentiating urgent/emergency events from normal events (this can be decided during implementation). 4. (OPTIONAL - if mode is PULL) At some moment of time, the Subscriber pulls an event: * The Subscriber makes a GET call to the Room service of the particular event type. * The Subscriber acknowledges receiving of event. 5. (OPTIONAL) The Subscriber requests event details. Some event details may have more restricted regulations for handling and may be not included in event type. In this case, the Subscriber requests these details directly from the publisher by making a GET call to the referenced Publisher with event id as a parameter. * This call implies the existence of an associated contract between the Subscriber and the Publisher. The Information Mediator Building Block creates a log of all messages published and distributed. If an event payload is very large, then it is recommended to just publish the "event" and let Subscribers get full details directly from the Publishers as needed. It will simplify event payloads, and reduce Turnaround time (TAT), storage, and bandwidth significantly. For example, “Clinic System A” wishes to broadcast data about a new birth so that it can be used to trigger asynchronous actions in several other systems. A Service is configured in Clinic System A that makes an HTTP POST request corresponding to the type of event and some valid JSON in the body to the Pub/Sub provider. The provider maintains a list of active “Subscribers” for that event type. Without manipulating the body, that message is effectively forwarded to each subscriber, so that REST services in the “Demographics Tracker”, “Insurance Registry”, and “Early Childhood Education” applications can consume that data and do things based on it. It should be noted that the shape of the payload will be agreed upon beforehand so that the responsibility of being able to “ingest” a “newBirth” payload from Pub/Sub will fall on the REST service provider. Once an event is ingested, the responsibility to deliver the payload lies with the Information Mediator “room” based on the importance given to the event by a Subscriber. Thus, during registration of the Subscriber, parameters such as max queue depth, retries, failsafe mechanisms, and error handling have to be configured by each Subscriber as they subscribe to an event type. # 10 Other Resources This section links to any external documents that may be relevant, such as standards documents or other descriptions of this Building Block that may be useful. ### 10.1 Key Decision Log [A historical log of key decisions regarding this Building Block](https://govstack-global.atlassian.net/wiki/spaces/GH/pages/183369729). ### 10.2 Future Considerations [A list of topics that may be relevant to future versions of this Building Block](https://govstack-global.atlassian.net/wiki/spaces/GH/pages/183500805).