Overview

Uptycs agent (osquery) needs cloud connectivity to function properly. However, endpoints might not be accessing the internet directly. A lot of scenarios have emerged where the endpoint uses proxy servers to connect to internet. This is specially true for secure environments, VDI environments etc. Osquery has the capability to connect to cloud via a configured proxy. But, that requires manual intervention. In many cases, there is a list of proxies that are provided to the end user (for example via a flags file) and it is tedious to manually select the best proxy configuration for osquery to reach the cloud. 

Solution

Proxy Discovery Steps

Osquery has been modified to find it's way to the cloud based on a sequence of auto-discovery steps. It is presumed that presence of a proxy server as a direct or indirect config parameter should be given precedence over establishing direct connection. This is similar to how browsers behave.

1. Before attempting send an enroll, config or distributed request to the cloud-server, osquery tries to check if cloud is reachable.
2. The first attempt to check if cloud is reachable is via the configuration stored in the database. This is empty on a new deployment but contains the last successful proxy configuration. This step is called the 'db'.
3. The second attempt is made via a proxy or list of configured proxies configured with the Operating System. Different OS have different proxy settings and hence different proxy discovery semantics. A separate section describes OS proxy discovery toward the end of this page. This is called the 'os' step.
4. If the attempt to connect the with the cloud via the OS-configured proxy fails, or if no OS-configured proxy is found, osquery reads the proxy_hostname flag. The flag may contain a single proxy or a list of proxies (separated by a comma but no space). Osquery tries to use the specified proxy or each proxy from the list to test connectivity to the cloud. This step is called the 'flags' step.
5. The first successful connection to the cloud via a proxy selected using above steps is considered the best proxy and is locally cached and stored into the database for subsequent connections to the cloud. This cache is used in the 'db' step and is used unless a connection to the cloud fails, in which case the above steps are re-executed to find a new best proxy server to connect to the cloud.
6. If connecting via the db, os, or flag steps fails, or if no configured proxy is found in os or flag steps, osquery tries to connect with the cloud directly. This step is called 'direct' step. A successful direct connection is also cached as a fact and is used in 'db' step.
7. If connection to the cloud is unsuccessful, attempts are made in increasing intervals till worker quits and is restarted by watcher.
   

Note : Password protected proxies / https proxies are not supported.

Configurable proxy discovery order

1. The order in which the above above four steps are executed are configurable via the osquery flag : proxy_auto_discovery_order . The default order is what is listed above and is written as
   --proxy_auto_discovery_order =db, os, flags, direct
2. The flag accepts only these four values and can be used to change proxy discovery order given a specific environment. However it is recommended to keep db as the first step always, otherwise every worker restart will trigger a proxy discovery.
3. The fact the proxy discovery happens is controlled via the flag :  enable_proxy_auto_discovery. Default is true. Disabling proxy discovery will cause only direct connections to be attempted.

Operating System proxy discovery

While the algorithm to select proxy configuration follows a sequence as described above, the algorithm to compute the proxy setting from the OS varies from OS to OS. This is the 'os' step above described in more detail:

On Windows:

1. Osquery first tries to check if "auto proxy detection" is enabled. if it is, it uses Windows API to make a mock request to Uptycs cloud and gets the proxy server in response.
   
2. It then checks from registry if proxy based on PAC file is enabled. A sample PAC file is attached to the confluence. If a PAC file URL is configured, it uses Windows API to run the PAC file against the Uptycs Server and gets the proxy url and port.

            A sample pac file looks like this:proxy.pac. Note: this works on Windows but not on Mac. We need to figure a good                 pac file for mac.

        3. It then checks for manual proxy configuration, and uses registry to get the conffigured proxy and port

On Linux:

Osquery only checks for HTTP_PROXY and HTTPS_PROXY environment variables in both upper and lower cases. In earlier version osquery would try to get proxy information from environment variables of other processes and use the first process that would show a valid value for http_proxy environment variable. This is no longer the case. Best way to set HTTP_PROXY environment variable on Ubuntu is to set it in /etc/default/osqueryd file.

On Mac

On Mac OS, osquery first tries to check if PAC file proxy is enabled. If it is, it uses NS APIs to get the proxy for Uptycs' cloud URL.


- It next checks for manually configured HTTP proxy. HTTPS proxy is not supported in Mac.


Both these settings can be toggled from System Configuration.

Though auto proxy configuration is possible in Mac, it is not supported for Osquery.