Authored by Bruce Chamberlin

Click to Expand Image

Three key areas:

  1. Data Transfer

  2. Data Transformation (Data Warehouse Processing)

  3. Reporting Data Warehouse

General

  1. All data is encrypted for transfer between all areas.

  2. No credentials for connection are stored in the code but all stored in the Azure Key Vault.

  3. We rely heavily on Azure security - but have architected it with security in mind. So we use the Azure key vault for all connection strings - we use encryption where ever possible.

  4. We use Visual Studio and Visual Team Services for source control and limit who can access source.

  5. We limit the users within our team that have access to the Azure data warehouse -  all users must use MFA to logon.

  6. We also use the Azure firewall to the database server to control what IP addresses have access to our Azure databases.

  7. We typically work from Azure VM’s and remote to these VMs to access any backend services. These are setup with all the standard Azure security checks and are based on the Azure base images.

Data Transfer

  1. The windows service pulls instructions and pushes data. It is scheduled to ‘wake up’ every x minutes (usually every 120 minutes) - connect back to our controller database with a unique security key for each instance and process any changes in the ConnectWise Manage or Automate database. It then pushes the changes in an encrypted and compressed set of XML files to the Azure Web Service. If you disable the Windows service - we will no longer get data. It is configured to use a read-only account to connect to your production ConnectWise DB. We support many versions of ConnectWise and so have done considerable work to make the Windows service data collection process tolerant of schema changes.

Data Transformation (DW Processing)

  1. Our system loads data into a data warehouse , transforms the data into a star schema. We make extensive use of Azure data factory to process the data. 

  2. Only limited staff members have access to these multi-tenanted data warehouses

  3. We use Azure SQL and this data is always encrypted at rest

Reporting Data Warehouse

  1. Azure Data Factory publishes the transformed data to a reporting data warehouse. There is a choice for each customer, as to where this reporting data warehouse is published - i.e. Multi-tenanted in our Azure Tenant, single tenant in your own Azure tenant (For options - ask us for a quote on your preference) . 

  2. In the case when the report data warehouse is published in your Azure tenant, you will have self service rights to read (and write) your data. 

  3. We use Power BI published into your O365 Power BI Service as the front-end for data insights. You can control who in your organization has access to each Power BI report. We use the Power BI rest API to publish or update our standard Power BI reports. To do this we need to register our publishing application into your Office 365 AD and need application rights granted to Power BI reports and datasets.


For queries, please contact support@cognition360.com