File carving provides forensic ability for security analysts to retrieve part or all of one or more files from customer endpoints. By default file carving is disabled. Customers will have to explicitly configure flags profile with disable_carver set to false to enable carving capability. Flag profile can be associated with a subset or all of customer endpoints.

Customer can select one or more endpoints and perform a real time query to carve a file. 

ExampleSELECT * FROM carves WHERE path = ‘/path/to/file’ AND carve = 1

The Uptycs endpoint agent (osquery) will carve data from the specific absolute file path and send the data to Uptycs back-end via HTTPS (TLS 1.2) POST request. File data is uploaded in multiple chunks (maximum chunk size = 5 MB). When the file size exceeds chunk size, multiple POST requests are sent from the endpoint to the back-end.

When the Uptycs back-end receives the first chunk (carve start):

• It creates a database entry with the carve information (file path, size and other metadata)

• It creates a unique encryption key that is specific to the file being carved

• Encryption key is encrypted with a master key and stored along with the carve metadata

• First chunk is uploaded to an S3 bucket using the encryption key as SSE-C via multi part upload 

For subsequent chunks same SSE-C encryption key is used while uploading chunks. For the last chunk, multi part upload complete S3 request is called for S3 to assemble all multi-upload parts and re-create the original content.

• Uptycs back-end DOES NOT save the chunks anywhere other than in the S3 bucket. As the chunks are received, they are uploaded to S3

• Uptycs DOES NOT retrieve the contents of the file or chunks from S3 bucket

• Customers can request Uptycs to use S3 buckets in their own AWS account. Uptycs will provide necessary IAM/S3 configuration for Uptycs back-end to write to carved files to customers S3 bucket (cross-account). This does not involve creating access/secret keys

• Permissions Uptycs needs for the S3 bucket are the following. 

  • s3:PutObject

  • s3:AbortMultipartUpload

  • s3:ListMultipartUploadParts

  • s3:DeleteObject

• Bucket can also be configured for Service Side Encryption (SSE), but is unnecessary since all contents will be encrypted with SSE-C

• Once carving is complete, customer can download the carved file using Uptycs UI or API. 

  • Download is available only as a one time operation. 

  • Once the download is requested, carve status is marked as DELETED in Uptycs back-end. SSE-C encryption key that is specific to the carve will be erased from the database. 

  • File cannot be downloaded again without carving from the endpoint again. 

  • Uptycs uses S3 API to generate a pre-signed URL for the carved file. 

  • This URL will directly point to S3 bucket and Uptycs does not proxy the download request. 

  • Once download is requested, Uptycs back-end waits 5 minutes and initiates delete request to delete the file from S3 bucket. 

  • If customer was not able to download the file in the 5 minute time frame, they will be required to carve again.  

• All requests to carve files and download carved files are logged in the API audit log table