falco/falcosidekick mTLS

Background

we are using falco to monitor unsafe actions in our cluster and pushing alerts and logs to sidekick to let sidekick pushing those to alertmanager and loki。 However, those connections are all plain connections. For security purpose, we have to enable mtls for all communications

Details

For falco => sidekick, first of all, it is supporting mtls since 0.36.0. You just need to enable mtls in http_output section on falco-config and provide the client cert/key to let server verifiy the client certificate.

refers to https://github.com/falcosecurity/falco/blob/master/falco.yaml#L693, what we need to proide is:

http_output:
   url: "https://sidekick-svc.ns.svc.cluster.local:listenport

   mtls: true

    ca_cert: "Path to the CA certificate that can verify the remote server. which means this ca signs server certificate, not the below client certs"

    client_cert: "/etc/ssl/certs/client.crt"

    client_key: "/etc/ssl/certs/client.key"

We only treat falco as client because falco only connects to sidekick actively.

So after falco, sidekick has to be configured also as server side

on falco-sidekick configuration https://github.com/falcosecurity/falcosidekick?tab=readme-ov-file#yaml-file

tlsclient:
  cacertfile: "/etc/certs/client/ca.crt" # CA certificate file for server certification on TLS connections, appended to the system CA pool if not empty, if your cert is signed by different ca, then procides ca that can verify server side certificate
tlsserver:
  deploy: true # if true, TLS server will be deployed instead of HTTP
  certfile: "/etc/certs/server/server.crt" # server certification file
  keyfile: "/etc/certs/server/server.key" # server key
  mutualtls: true # if true, mTLS server will be deployed instead of TLS, deploy also has to be true
  cacertfile: "/etc/certs/server/client_ca.crt" # for client certification if  mutualtls is true
  notlsport: 2810 # port to serve http server serving selected endpoints (default: 2810)
  notlspaths: # if not empty, and tlsserver.deploy is true, a separate http server will be deployed for the specified endpoints
    - "/ping"

And then start falco and falcosidekick, then falco should be able to push to sidekick in mTLS way

For sidekick => am/loki, we need to configure am and loki as server side, but this passage wont deal with that because out of scope

for sidekick, in same config file:

mutualtlsfilespath: "/etc/certs" # folder which will used to store client.crt, client.key and ca.crt files for mutual tls for outputs, will be deprecated in the future (default: "/etc/certs")
mutualtlsclient: # takes priority over mutualtlsfilespath if not emtpy
  certfile: "/etc/certs/client/client.crt" # client certification file
  keyfile: "/etc/certs/client/client.key" # client key
  cacertfile: "/etc/certs/client/server_ca.crt" # for server certification

Conclusion

Enabling mtls is always a better way for falco to secure the connection because sometimes some sensitive infomation may get pushed and hajacked but unknown authority if using plain HTTP.

Notes

To verify if a cert is signed by a ca, we can use

openssl verify --verbose --CAfile <ca.crt> --cert <certification.crt>

If above returns ok, then it means the certification.crt is signed by ca

imjliu