How data is collected by your platform?
The Recorder, which is a tool that you deploy in your infrastructure, generates data files which must be submitted to our platform:
- Scans (
.kss): Generated when you use the Scanner on a new release of your application, which will allow for the detection of changes between all of your releases.
- Test Footprints (
.ksa): Generated when you execute some tests and record their runtime footprint (Test Footprint) using our Agent.
These data files are first stored on your servers, then submitted to our platform.
When data is collected?
Whenever a new release (build) is available for testing, you run a Scan it to identify changes.
During testing, a Test Advisor Agent that is installed on your test server will record your Test Footprints.
How is data submitted to your platform?
You always have control over data transmissions because they are never transmitted automatically by our tools. You are in charge of collecting data files (
.ksa) on the Recorder and uploading them on our platform. You can upload the files manually through the Recorder user interface (a part of the Controller) or in an automated way through the Web API.
Files are always uploaded using a secure communication (HTTPS) to the same entry point: https://cockpit.kalistick.com.
What is stored in the data files?
A Scan file is generated when you scan your application. It stores synthesis information about every file of your deployed application. The objective is to identify files, and even lines of code, that have changed between two versions of your application.
The following information is stored in a Scan file:
- The relative path of all files used by your application runtime.
- The compiled code dictionary (for Java, C# or VB files):
- Full name of each class, for example:
- Relative physical path of each class, for example:
- Signature of each method, for example:
String myMethod(String myParam)
- Locations of method within their class file (for example, starts at line 25 and finishs at line 53)
- Full name of each class, for example:
- Signature of file contents
This is the key data used to detect changes between your application versions: A checksum (MD5 or SHA-1) is stored for each application file.
For binary files (images, third-party libraries, and so on), the signature is computed from the entire file contents. For text files and compiled files related to the own code of the application (Java, C# or VB classes), a signature is computed for each line of code.
A signature cannot be reverted; it is impossible to find out the original content. However any change in a line of code will generate a change in the signature. Our platform compares all your application signatures to identify any changes and identify the impact of these changes without any details about the code itself.
Note: A salt can be configured for checksum generation.
A Test Footprint file is generated when you record your tests using a Test Advisor Agent. They store information about what is executed by each test.
A Test Footprint file stores the following information:
- General information about all recorded tests: Identifier, name, execution status, issue identifiers (if an issue is created and linked to the test case), date of execution (start/end)
- URL(s) used during each test to navigate within your application (only for Web applications); for example:
- Method call trees: Signature of all methods executed by each test, with the full execution tree, that is, with all ancestor calls.
- Covered lines: For each test, lines of compiled code (Java, C# or VB) which have been executed and marked as covered for the test.
Can I audit and control the submitted data?
Data files generated by the Recorder can always be controlled. The Scan file (
.kss) is actually a ZIP file that you can open and browse using a standard ZIP tool. The Test Footprint file (
.ksa) has a highly optimized binary format based on a proprietary algorithm, but we provide a parser to verify its content.
Where is the hosting and data storage?
We are using Amazon Cloud infrastructure (AWS), the leading Cloud provider within several regions and available around the world. We use best practices from Amazon to protect our servers, using the security infrastructure provided by Amazon, but also additional layers to guarantee the best protection.
Who have access to these data?
All account data is accessible by Coverity employees and contractors, all of whom are under strict confidentiality agreements.
Is there any on-premises option?
Yes, there is. If you cannot use a Cloud infrastructure, you can have your own instance of the Test Advisor Cockpit.