Redis with TLS in-transit support

Support Kolibri
1

If you have a specific technical inquiry, construct your post with the following:

Summary

Cannot seem to connect to a Redis server with TLS to have encryption in-transit.

Technical details

  • Kolibri version 0.15.7

We are hosting a Kolibri environment in AWS. We configured two different Redis instances (not clustered) running Redis 6.2.6. One of them is configured to use TLS for encryption in-transit, and one is not.

When setting the CACHE_LOCATION to the one with encryption in-transit we found that Kolibri just hangs. We tried things like (replacing the actual hostname with “endpoin”):

CACHE_LOCATION = endpoint:6379
CACHE_LOCATION = tls://endpoint:6379

But it did not seem to connect. When we just connected to the one without encryption in-transit things started fine.

Does Kolibri not support connecting to Redis over TLS, or is there a config requirement we’re just not setting correctly.

We are able to connect and get a response to the encrypted instance when we use redis-cli:

redis-cli -h endpoint --tls -p 6379 PING

2

Under the hood, Kolibri is using the Django Redis Cache backend. Looking at the documentation I can find no mention of configuration for encryption in-transit, so it may be a limitation of the cache backend we are using.

I only did a very rudimentary search, so possible I missed something though!

3

From what I’m seeing, it seems that CONNECTION_POOL_KWARGS needs to be configured to set this, but I am not seeing an option for this in the options.ini file. Is it not possible to specify this setting?

4

@Brian_LePore No there currently isn’t an option to specify that in Kolibri’s options.ini. With the version of the Python redis package that Kolibri uses, a path ssl_ca_certs would also be required.

5

It might be easiest to make your own Django settings file to accomplish this and set the cache in a way that is controlled more directly by you.

The file would just import the base Kolibri settings:

from kolibri.deployment.default.settings.base import *
...

Then define a custom CACHES setting.

You can then use the DJANGO_SETTINGS_MODULE environment variable to set the Python import path for your settings file.

6

So I made a directory/file located at /var/kolibri/custom/cache.py with the following content.

from kolibri.deployment.default.settings.base import *

CACHES = {
    'default': {
        "BACKEND": "redis_cache.RedisCache",
        "LOCATION": cache_options["CACHE_LOCATION"],
        # Default time out of each cache key
        "TIMEOUT": cache_options["CACHE_TIMEOUT"],
        "OPTIONS": {
            "PASSWORD": cache_options["CACHE_PASSWORD"],
            "MAX_ENTRIES": cache_options["CACHE_MAX_ENTRIES"],
            # Pin pickle protocol for Python 2 compatibility
            "PICKLE_VERSION": pickle_protocol,
            "CONNECTION_POOL_CLASS": "redis.BlockingConnectionPool",
            "CONNECTION_POOL_CLASS_KWARGS": {
                "max_connections": cache_options["CACHE_REDIS_MAX_POOL_SIZE"],
                "timeout": cache_options["CACHE_REDIS_POOL_TIMEOUT"],
                "skip_full_coverage_check": True,
                "ssl_cert_reqs": None,
                "ssl": True,
                "ssl_ca_certs": "/etc/ssl/certs/ca-certificates.crt"
            },
        },

    }
}

I then set the environment variable: export DJANGO_SETTINGS_MODULE=/var/kolibri/custom/cache.py

When I tried restarting kolibri I received the error:

ModuleNotFoundError: No module named '/var/kolibri/custom/cache'

I feel like I’m close and just missing something. Did I set the file wrong?

7

So I renamed the file to kolibri.py and moved it to "/usr/lib/python3/dist-packages/kolibri/kolibri.py and set my export to “kolibri.kolibri” (I am knew to Python/django stuff so I’m not that familiar with how things should be set) and once I did that I found my file was at least being referenced.

Changed the file to:

from kolibri.utils.conf import OPTIONS
from kolibri.deployment.default.settings.base import *
...

cache_options = OPTIONS["Cache"]

pickle_protocol = OPTIONS["Python"]["PICKLE_PROTOCOL"]

CACHES = {
    'default': {
        "BACKEND": "redis_cache.RedisCache",
        "LOCATION": cache_options["CACHE_LOCATION"],
        # Default time out of each cache key
        "TIMEOUT": cache_options["CACHE_TIMEOUT"],
        "OPTIONS": {
            "PASSWORD": cache_options["CACHE_PASSWORD"],
            "MAX_ENTRIES": cache_options["CACHE_MAX_ENTRIES"],
            # Pin pickle protocol for Python 2 compatibility
            "PICKLE_VERSION": pickle_protocol,
            "CONNECTION_POOL_CLASS": "redis.BlockingConnectionPool",
            "CONNECTION_POOL_CLASS_KWARGS": {
                "max_connections": cache_options["CACHE_REDIS_MAX_POOL_SIZE"],
                "timeout": cache_options["CACHE_REDIS_POOL_TIMEOUT"]
            },
            "CONNECTION_POOL_KWARGS": {
                #"skip_full_coverage_check": True,
                #"ssl_cert_reqs": None,
                "ssl": True,
                "ssl_ca_certs": "/etc/ssl/certs/ca-certificates.crt"
            },
        },

    }
}

In the options.ini file I prefixed the connection with rediss:// so it looked like:

CACHE_LOCATION = rediss://endpoint:6379

But that seemed to not help. Ultimately I needed to manually make changes following this merge.

Now Kolibri will start, but I am unsure how to verify that this is working. Any suggestion?