ITENTIAL ADAPTERS TECHNICAL RESOURCE
Adapter Properties
General Configuration
- Properties exist to allow flexibility in the adapters. Changes to properties can be made through the Itential Application Platform (IAP) user interface (UI). When these changes are made, the adapter will be restarted but IAP will remain up. The ability to refresh adapter properties using refreshProperties does exist, but IAP does not currently make use of that capability.
- The adapter properties are defined in the propertiesSchema.json file. This is a json schema file that can provide information such as the property data type (String, Integer, Boolean, Enumeration, etc.), a description of the property, and whether there is a default value.
- You can add additional properties to the propertiesSchema.json file, if needed. The adapter properties that are originally defined in the propertiesSchema.json file are utilized by the adapter-utils libraries. Removing or changing any of the default values for these properties could cause the adapter to not function properly.
- Adapter properties can be overridden on an individual call through the callProperties defined in the request object. For more on how to do this see the presentation on Adapter Code (adapter.js).
- Information about adapter properties can also be found in the README.md file for every adapter.
Base Properties
Definition
- host (string): Hostname or IP address of the external system server. This property is required. Without it, the adapter will not be able to perform its essential function of communicating with another system.
- http://xyz.abc.com:8080/xyz/abc
- port (number): Port on which to connect to the external system. This property is required. Without it, the adapter will not be able to perform its essential function of communicating with another system. Even if you are using the default ports be sure to verify the port number in this property.
- When a URL appears as http://xyz.abc.com/xyz/abc,
it is utilizing the default port. - HTTP default is 80.
- HTTPS default is 443.
- When a URL appears as http://xyz.abc.com/xyz/abc,
- base_path (string): A base path appears in most, if not all, of the API calls. You can override the base path on individual actions as needed. Using base path makes it easier to maintain the adapter whenever changes are made to the system path. If it changes to /api/rest, then you only need to change the property instead of having to change the entitypath on every action in the adapter.
- http://xyz.abc.com:8080/api/rest/v1/abc
- version (string): Version is like base path in that it appears in most, if not all, of the API calls. You can override the version on individual actions as needed. Using version makes it easier to maintain the adapter whenever changes are made to the API version. If it changes to v1.5, then you only need to change the property instead of having to change the entitypath on every action in the adapter.
- http://xyz.abc.com:8080/api/rest/v1.5/abc.
- cache_location (enum): Determines where the entity cache will be stored. Some brokers will check if an adapter can perform an action on an entity prior to requesting the action be done. In those scenarios, it is more efficient for the adapter to maintain entity lists in cache. If the adapter you are building has no need to cache basic entity lists, do not use caching as there is some overhead involved. The following values are supported:
- none – No caching.
- local – In memory (cache lost if adapter restarts).
- redis – In Redis (preserved across adapter restart – slightly less performant).
- save_metric (boolean or string): The adapter now collects metrics on each call. These metrics automatically provide information on the individual call. If save_metric is turned on, additional metrics will be maintained so that over time you will be able to see the success rates and average times for every action within the adapter. If a string is provided, the adapter will create a storage directory within the adapter and assume that is the path to where the metrics are to be stored.
- stub (boolean): This flag determines whether the adapter should run standalone with mock data or integrated to the external system. When the adapter is deployed this should almost always be false. It can be set to true when running unit tests, performing integration tests with mock data, or when the adapter is running integrated with IAP to test capabilities like workflow but not connected to the other system.
- protocol (enum): The protocol the adapter should use to communicate to the other system. The adapter currently supports the following protocols:
- HTTP
- HTTPS
SAMPLE PROPERTIES
"host": "localhost",
"port": 443,
"base_path": "/api",
"version": "v1.16.0",
"cache_location": "none",
"save_metric": true,
"stub": true,
"protocol": "https",
Example
- In this example, the adapter wants to communicate with a system that is at mysystem.abc.com.
- It uses the HTTPS protocol on port 3443.
- All requests to this system start with /api/rest.
- The system is currently running v2.3 of its API, and the action paths should look like this: https://mysystem.abc.com:3443/api/rest/v2.3/specificpathforaction
- There is no caching of entities.
- Metrics will be saved for the request
- This example will run in production mode and communicate with the other system.
SAMPLE PROPERTIES
"host": "mysystem.abc.com",
"port": 3443,
"base_path": "/api/rest",
"version": "v2.3",
"cache_location": "none",
"save_metric": true,
"stub": false,
"protocol": "https",
Authentication Properties
Definition
All authentication properties are within the authentication object within the properties. If using request_token, there is also an action (getToken) defined in the .system entity that is used to define the token request.
- auth_method (enum): Required field. The authentication method used for the external system. The following methods are supported:
- basic user_password: Authentication will always require a username and password.
- static_token: Authentication is always done with a static token provided in the token property.
- request_token: Authentication is initially done with a username and password which will return a token that can be used in subsequent requests.
- no_authentication: No authentication is required to talk to the system or authentication will be done outside of the adapter run time libraries.
- username (string): Required for basic user_password and request_token. The username for authenticating with the other system
- password (string): Required for basic user_password and request_token. The password for authenticating with the other system. It can be encrypted through the adapter encryptProperty method or through IAP. Note: You must use encyptProperty of the adapter that will be specified using the property.
- token (string): Required for static_token only. The token to use when authenticating with the external system. This is only used when the authentication is static token and thus the token is known.
- token_timeout (integer): How long a token is valid before it will timeout when using dynamic tokens. The system will request a new token prior to the defined timeout. Special values include:
- -1 – Always get a new token.
- 0 – Use the expiration time returned with the token. Must be defined in the token schema response and have an acceptable format.
- token_cache (enum): When using dynamic token, this tells the adapter where to store the token until it expires.
- Local – In memory. Lost if adapter restarts.
- Redis – Preserved across adapter restart. Slightly less performant.
- invalid_token_error (integer): Error returned when a request is rejected with invalid token. Adapter will request new token.
- auth_field (string): Required. The field in the request to the external system where the authentication information is placed. For example, if the system expects it in header, then headers and then Authorization, this field should be set to header.headers.Authorization.
- auth_field_format (string): Required. The format of the field in the request that contains the authentication information. You can specify a combination of text and dynamic information.
- For example, if you have b64 encoding for Basic Auth, this property may look like: {b64}Basic {username}:{password}{/b64}.
- This tells the adapter to b64 encode a string that will look like “Basic myuser:mypasswd”.
- You can also combine text with tokens by setting this property to Token {token} or Bearer {token}.
SAMPLE PROPERTIES
"authentication": {
"auth_method": "no_authentication",
"username": "username",
"password": "password",
"token": "",
"token_timeout": 1800000,
"token_cache": "local",
"invalid_token_error": 401,
"auth_field": "header.headers.X-AUTH-TOKEN",
"auth_field_format": "{token}"
},
Example
- In this example, mysystem.abc.com uses a two-step authentication process.
- In this first step, you will authenticate with a username (IAP) and password (isTheBest!).
- The system will then return a token that you need to put in every subsequent request.
- On subsequent requests, the token will be placed in the header field called My-Special-Token. The format of the authentication field is ”My Token is xxxx”
- The response token that’s received is valid for 10 minutes (600s – 600000ms)
- When the authentication fails or the token is no longer valid, you will receive a 401 error on the request.
SAMPLE PROPERTIES
"authentication": {
"auth_method": "request_token",
"username": "IAP",
"password": "isTheBest!",
"token": "",
"token_timeout": 600000,
"token_cache": "local",
"invalid_token_error": 401,
"auth_field": "header.headers.My-Special-Token",
"auth_field_format": "My Token is {token}"
},
Authentication Call
- The call to get the token is: https://mysystem.abc.com:3443/api/rest/v2.3/login. The host, port, base_path, and version are defined in base properties. So the path defined here is just /login.
- It is a POST call as it has a body. The body sent to the other system contains two fields, user and passwd. The fields username and password are the IAP names for those fields.
- The response from the other system contains the field for access_token. This is what IAP calls the token and will use it in subsequent requests.
- For more authentication examples see the presentation on Adapter Authentication.
entities/.system/action.json
{
"name": "getToken",
"protocol": "REST",
"method": "POST",
"entitypath": "{base_path}/{version}/login",
"requestSchema": "tokenReqSchema.json",
"responseSchema": "tokenRespSchema.json",
"timeout": 0,
"sendEmpty": false,
"requestDatatype": "JSON",
"responseDatatype": "JSON",
"headers": {},
"responseObjects": [
{
"type": "default",
"key": "",
"mockFile": "mockdatafiles/getToken-default.json"
}
]
},
PARTIAL REQUEST SCHEMA
{
…
"username": {
"type": "string",
"description": "username to log in with",
"external_name": "user"
},
"password": {
"type": "string",
"description": "password to log in with",
"external_name": "passwd"
}
},
"required" : ["username","password"],
…
}
PARTIAL RESPONSE SCHEMA
{
…
"token": {
"type": "string",
"description": "the token returned from system",
"external_name": "access_token"
}
},
…
}
Healthcheck Properties
Definition
All healthcheck properties are within the healthcheck object within the properties. There is a “healthcheck” action defined in the .system entity that is used to define the healthcheck request.
- type (enum): Required field. The type of healthcheck to use.
- none – Do not run any healthchecks.
- startup – Run a healthcheck at start up only
- intermittent – Run healthchecks at a defined interval.
- frequency (integer): Required if intermittent. The frequency in which to run intermittent healthchecks.
SAMPLE PROPERTIES
"healthcheck": {
"type": "startup",
"frequency": 60000
},
Example
- In this example, a healthcheck is set to run every 5 minutes (300s – 300000ms).
- The call for the healthcheck is a GET method: https://mysystem.abc.com:3443/api/rest/v2.3/getSystemStatus The host, port, base_path, and version are defined in base properties. So the path defined here is just /getSystemStatus.
- The data in the response is not the primary focus here, just that we got a response. For this example, we can use the default schema with no translation.
SAMPLE PROPERTIES
"healthcheck": {
"type": "intermittent",
"frequency": 300000
},
entities/.system/action.json
{
"name": "healthcheck",
"protocol": "REST",
"method": "GET",
"entitypath": "{base_path}/{version}/getSystemStatus?{query}",
"requestSchema": "schema.json",
"responseSchema": "schema.json",
"timeout": 0,
"sendEmpty": false,
"requestDatatype": "JSON",
"responseDatatype": "JSON",
"headers": {},
"responseObjects": [
{
"type": "default",
"key": "",
"mockFile": "mockdatafiles/healthcheck-default.json"
}
]
}
Request Properties
Definition
All request properties are within the request object within the properties. These tell the adapter libraries how to handle details that come up on a request.
- number_redirects (integer): If the adapter should allow redirects, this property tells the adapter how many redirects to allow before returning an error.
- number_retries (integer): Required if the request object. The number of times to retry a request that was either aborted or failed with a retry error before returning an error.
- limit_retry_error (integer or array): Required if the request object. An error that has been defined as one the adapter should re-attempt. Supports an array of integers or an array containing ranges (e.g., [401,“500-599”]).
- failover_codes (array of integers): List of error codes used by the adapter to send an error response that notifies the platform to attempt the request through another adapter, if available.
- attempt_timeout (integer): Required if the request object. Defines the time in milliseconds (ms) after which the adapter should abort a request and retry after the external system is available.
- global_request (object): Provides a way to have global data used in all requests the adapter makes. It is key that all values are static. Dynamic values are handled through adapter.js.
- payload (object): If every payload sent to the other system is supposed to contain specific fields with static values, that part of the payload can be set here.
- uriOptions (object): If every request sent to the other system is supposed to contain query fields with static values, those options can be defined here.
- addlHeaders (object): If every request sent to the other system is supposed to contain specific headers with static values, those headers can be defined here.
- authData (object): If every request sent to the other system is supposed to contain authentication data with static values, that data can be defined here.
- healthcheck_on_timeout (boolean): Required if the request object. Defines whether a healthcheck should be performed when a request has timed out to make sure the other system is still available.
- return_raw (boolean): Tells the adapter runtime libraries to return the raw response along with the results being returned. This is helpful to collect mock data but may also be required on some calls.
- archiving (boolean): Deprecated. Do not use except in a lab environment for testing the adapter.
SAMPLE PROPERTIES
"healthcheck": {
"type": "intermittent",
"frequency": 300000
},
entities/.system/action.json
{
"name": "healthcheck",
"protocol": "REST",
"method": "GET",
"entitypath": "{base_path}/{version}/getSystemStatus?{query}",
"requestSchema": "schema.json",
"responseSchema": "schema.json",
"timeout": 0,
"sendEmpty": false,
"requestDatatype": "JSON",
"responseDatatype": "JSON",
"headers": {},
"responseObjects": [
{
"type": "default",
"key": "",
"mockFile": "mockdatafiles/healthcheck-default.json"
}
]
}
Example
- In this example, there are no redirects.
- The adapter will retry up to 5 times when the request times out.
- In this example, there is no retry on error.
- There is no failover to other adapters.
- If we have not received a response on the request after 1 minute (60s – 60000ms), the system will abort and resend the request.
- Every call to this system should have a header of “SimonSays” with a value of “Please”.
- When there is a timeout (no response on a request for 1 minute), we will run a healthcheck to make sure the system is still responding.
SAMPLE PROPERTIES
"request": {
"number_redirects": 0,
"number_retries": 5,
"limit_retry_error": 999,
"failover_codes": [],
"attempt_timeout": 60000,
"global_request": {
"payload": {},
"uriOptions": {},
"addlHeaders": {
"SimonSays": ”Please",
},
"authData": {}
},
"healthcheck_on_timeout": true,
"return_raw": false,
"archiving": false
},
SSL Properties
Definition
All SSL properties are within the SSL object. These tell the adapter libraries how to handle security related to the request.
- ecdhCurve (string): In testing with Node.js 8 and Node.js 9 there is an issue sometimes where a PROTO error is returned when attempting to connect. This issue was fixed in later versions of Node but to fix it in Node.js 8 and Node.js 9, set ecdhCurve to “auto” in the property file. Note: This is the only use of this property.
- enabled (boolean): Required in SSL object. Determines whether the external system has SSL enabled for requests.
- accept_invalid_certs (boolean): Required if SSL enabled. Whether the adapter should accept (i.e., ignore) invalid certificates. Note: Only set this to true in lab environments.
- ca_file (string): Required if SSL enabled and not accepting invalid certs. The fully qualified path name to the ca_file used for SSL.
- key_file (string): The fully qualified path name to the key_file used for SSL.
- cert_file (string): The fully qualified path name to the certificate file used for SSL.
- secure_protocol (string): The secure protocol for the SSL handshake.
- ciphers (string): Required if SSL enabled. The hyphenated list of acceptable ciphers.
SAMPLE PROPERTIES
"ssl": {
"ecdhCurve": "",
"enabled": false,
"accept_invalid_cert": true,
"ca_file": "",
"key_file": "",
"cert_file": "",
"secure_protocol": "",
"ciphers": ""
}
Example
- The system is secured with SSL. Thus, SSL needs to be enabled.
- For lab environments, it is permissible to just enable SSL and then accept invalid certificates.
- Note: This practice should only be used in labs.
- For production, you will want to provide other information.
- In this case, the CA file is in /root/mycafile.
- The secure protocol will be
SSL 3.0. - In this case, no ciphers are used.
SAMPLE PROPERTIES
"ssl": {
"ecdhCurve": "",
"enabled": true,
"accept_invalid_cert": false,
"ca_file": "/root/mycafile",
"key_file": "",
"cert_file": "",
"secure_protocol": "SSLv3_method",
"ciphers": ""
}
Proxy Properties
Definition
All proxy properties are within the proxy object. These tell the adapter libraries how to handle proxied requests to another system.
- enabled (boolean): Required for the proxy object. Determines whether there is a proxy between Itential and the other system.
- host (string): Required if proxy enabled. The host information for the proxy server
- port (integer): Required if proxy enabled. The port information for the proxy server.
- protocol (enum): The protocol to use for the proxy. The following protocols are supported:
- HTTP
- HTTPS
- Socks4
- Socks5
ACTION.JSON
"proxy": {
"enabled": false,
"host": "",
"port": 1,
"protocol": "http"
},
Example
- The system is accessed through a Proxy Server.
- The proxy server is at: https://myproxy.xyz.com:443
ACTION.JSON
"proxy": {
"enabled": true,
"host": "myproxy.xyz.com",
"port": 443,
"protocol": "https"
},
Throttle Properties
Definition
All throttle properties are within the throttle object. These tell the adapter libraries how to handle throttling of requests to the other system.
- throttle_enabled (boolean): Required for throttle object. Turns throttling on for the adapter.
- number_pronghorns (integer): Number of Itential platforms over which the throttling queue needs to work.
- sync_async (string): Not currently used; reserved for future use.
- max_in_queue (integer): The maximum number of requests the adapter should hold in the throttling queue. Requests over the maximum will be rejected immediately to prevent the queue from growing without bound.
- Note: The Itential platform can hold thousands of items in the queue, but how fast they get responses will depend on the other system.
- concurrent_max (integer): The maximum number of sent requests that can be outstanding with the external system at any given point in time.
- expire_timeout (integer): Defines the time in milliseconds (ms) after each request is sent that the adapter needs to wait before sending the next request. This allows time for a request slot or license to become available again in the other system.
- avg_runtime (integer): Defines the time in milliseconds (ms) that the adaper should expect the external system to respond. This number is used as a seed time for determining queue waits. It is replaced by average run time of the last 25 requests as the adapter processes those requests.
- priorities (array): Defines priority rules for handling requests within the throttle queue. This allows you to self-define a small or large number of rules for prioritization.
- value (integer): Identifies a prioritization rule configuration. This value will also be provided in the request object priority so that the adapter knows which prioritization rule to apply to the request.
- percent (integer): Defines the percent of the queue that comes before this priority. For example, 0% will result in the request being put first in the queue, and 25% means the request will be put 25% of the way through the queue. Thus, if there are 100 requests in the queue, the new request will be placed at 25.
SAMPLE PROPERTIES
"throttle": {
"throttle_enabled": false,
"number_pronghorns": 1,
"sync_async": "sync",
"max_in_queue": 1000,
"concurrent_max": 1,
"expire_timeout": 0,
"avg_runtime": 200,
"priorities": [
{
"value": 1,
"percent": 25
}
]
},
Example
- The system can only handle five (5) concurrent requests.
- There is only one IAP running with this adapter.
- Whenever the queue has 500 requests, all other requests should be rejected to indicate the queue is full so that the adapter has a chance to catch up with the other requests.
- Each request to the system should take approximately 400ms.
- Approximating this is ok as it is only used as an initial seed.
- The adapter utilizes actual call averages while the calls are in progress.
- Three different priority queues should be created for requests.
- The 1st is for blocking issues that should be at the front of the queue.
- The 2nd is for requests which are still critical but not critical enough to be at the front of the line.
- The 3rd is for requests that are not critical but should be prioritized above the normal usage.
- If there is no priority with the request it will be at the end of the throttle queue.
SAMPLE PROPERTIES
"throttle": {
"throttle_enabled": true,
"number_pronghorns": 1,
"sync_async": "sync",
"max_in_queue": 500,
"concurrent_max": 5,
"expire_timeout": 0,
"avg_runtime": 400,
"priorities": [
{
"value": 1,
"percent": 0
},
{
"value": 2,
"percent": 10
},
{
"value": 3,
"percent": 50
}
]
},
Priority Queues
- The 1st priority is for blocking issues and these items need to be at the front of the queue.
- When there is a priority 1 request, it will be matched to the first rule and put at the front of the throttle queue.
- The 2nd priority is for requests which are still critical but not critical enough to jump to the front of the line; they are put in the queue at 10% – in front of 90% of the current requests.
- When there is a priority 2 request, it will be matched to the second rule and be put in the throttle queue after the first 10% of the requests.
- The 3rd priority is for request that are not critical to current business but should be prioritized above the normal usage; these are set at 50%.
- When there is a priority 3 request, it will be matched to the third rule and put in the throttle queue after the first 50% of the requests.
- If there is no priority with the request it will be at the end of the throttle queue (100%).
Mongo Properties
Definition
The Mongo properties are used to define a database where the adapter can store information. At the current time, two things are stored in the database – metrics and throttling queue.
- host (string): The server where the adapter mongo database can be found.
- port (integer): The port on the server where the adapter mongo database can be found.
- database (string): The adapter database within the mongo instance.
- username (string): The username used to log into the database.
- password (string): The password used at login to the database.
- replSet (string): If a replica set is being used, it should be defined here.
- db_ssl (boolean): Defines SSL encryption with the database.
- enabled (boolean): Determines whether the mongo server system has SSL enabled for requests.
- accept_invalid_certs (boolean): Determines if the adapter should accept invalid (i.e., ignore) certificates. Note: Only set this to true in lab environments.
- ca_file (string): Required if SSL enabled and not accepting invalid certificates. The fully qualified path name to the CA file used for SSL.
- key_file (string): The fully qualified path name to the key file used for SSL.
- cert_file (string): The fully qualified path name to the certificate file used for SSL.
SAMPLE PROPERTIES
"mongo": {
"host": "mymongo.com",
"port": 27017,
"database": "adapter-xyz",
"username": "username",
"password": "password",
"replSet": "replica_set",
"db_ssl": {
"enabled": false,
"accept_invalid_cert": true,
"ca_file": "",
"key_file": "",
"cert_file": ""
}
},
Example
- Keep in mind the adapter does not need Mongo to work. You can either turn metrics off, or the adapter can store metrics on the file system and throttling can be done in memory.
- The adapter does not want to store database information in the IAP database because it can cause issues when upgrading IAP.
- The example properties show an adapter database on server mymongo.com accessible on port 27017. The adapter database is called adapter-xyz and we are connecting to the database via username and password.
- In this example:
- A replica set is not used.
- SSL is not used to connect.
SAMPLE PROPERTIES
"mongo": {
"host": "mymongo.com",
"port": 27017,
"database": "adapter-xyz",
"username": "username",
"password": "password",
"replSet": "",
"db_ssl": {
"enabled": false,
"accept_invalid_cert": true,
"ca_file": "",
"key_file": "",
"cert_file": ""
}
},
Depreciations
These are properties that should no longer be used. Although they are supported in the code base, it is not a best practice to use them. Knowing about them is helpful since some adapters may still use them instead of the .system entity. These properties were replaced with the .system entity for authentication and healthcheck. The .system entity allows for more expanded capabilities in the token and healthcheck calls.
Authentication
- token_user_field (string)
- Identifies the user credential when requesting a token.
- token_password_field (string)
- Identifies the password credential when requesting a token.
- token_result_field (string)
- Will contain the token when requesting a token.
- token_URI_path (string)
- Used during authentication to retrieve a dynamic token.
Healthcheck
- protocol (string)
- Defines the protocol (i.e., REST) used to communicate with the other system.
- URI_Path (string)
- The call path the healthcheck should make.