Support for other databases

Support Kolibri
1

Summary

How can I run Kolibri on a different database on linux (mint). I have been having issues with the sqlite database that became corrupt (I/O errors) after many requests. I feel postgres is a better database in general for this kind of application and want to change it. I found the DJANGO_SETTINGS_MODULE setting /etc/kolibri/daemon.conf, but how do I create my own module to set the django settings? I don’t want to start messing around with the python package installed by kolibri, as running apt upgrade might overwrite it in that case. Any clues on how I can create my own Django settings for Kolibri? The Kolibri docs are not to clear on this (I’d be happy to contribute).
Thanks a lot for the help.

Technical details

  • Kolibri version: 0.12
  • Operating system: Linux Mint 19.1
  • Browser: Firefox
2

Hi @dolf
First of all, I would like to know if you can provide more info on the issues you had with sqlite. Is there any log you can provide or steps to reproduce them? 0.12.0 version of Kolibri shoud have solved all the past known issues in this field. It’s important to fix this kind of problem and your help would be appreciated.

Secondly, kolibri can be run using postgresql, the setup is done editing the options.ini file that exists (or you can create if it does not exist) inside the Kolibri home directory.

You need to setup a postgresql database, and with its data (hostname, database name, username and password) write this infomation in the options.ini file:

    [Database]
    DATABASE_ENGINE="postgres"
    DATABASE_NAME=your_db_name
    DATABASE_HOST=your_db_host (probably "localhost")
    DATABASE_USER=your_db_user
    DATABASE_PASSWORD=your_db_user_password

Hope this helps
José L.

3

Hi @jredrejo ,

Thanks for the help. That is useful info. I found the docs on the options.ini file, but the problem is the docs don’t list what options I can specify in there other than the http_port and profiling. Are the options for the options.ini file documented somewhere?

Right now I don’t have the logs available as this is not my server. I just came in to help a school, and left already. But next time I’m there I will get them. To be fully honest, I “think” it is Kolibri 0.12 (it was installed recently), but am not 100% sure. I will check next time as well.
But off the top of my head this is what I remember:
A few requests at a time are fine, but if we fire a whole bunch of requests at the same time (I ran 1000 requests from a python script), especially while streaming some movies, Django logs an error in .kolibri/server.log that there was an I/O error, and I think also an error that the SQLite DB is not a file or encrypted. As a result Kolibri throws a 500 error (of course). If we restart Kolibri, the error disappears.

4

hi @dolf I have checked there is not any documentation on how to use Kolibri with Postgresql.
You can see the parameters I recommended in the source code https://github.com/learningequality/kolibri/blob/develop/kolibri/utils/options.py#L47

I will fill an issue for the documentation. Using Postgresql is clearly not something to be done by a non-skilled person. Maybe that’s the reason why it’s not documented yet, to avoid messing the average user. In anycase, it would be good to have it written.

About the database problem you had, version 0.12.0 was released on March, 4th. So if you did your tests before that date you were using a previous version where those issues were reported. If you have a chance to do the same tests after upgrading to the latest version of Kolibri It would be really helpful.

José L.

5

Awesome, thanks for the link. That is enough for me. But I’d be happy to contribute docs if you want. I am probably going to setup kolibri with postgresql and nginx so we can have multiple workers serve requests and make efficient use of the multiple cores in the server and potentially have nginx also serve static files (didnt check yet if that is a viable option). But that is indeed not a normal user’s situation. But if there is interest, I’d be happy to contribute docs on how to get this setup (through a pull request on your docs?).

I will also check back on the server that I was testing on what version it is, get the logs and if not 0.12, update it and test again, before changing over to postgres/nginx.

6

Hi @dolf
if you want to do a nginx setup and like to be a tester I’d recommend you to take a look at the kolibri-server package. It automatically setups nginx & uwsgi to do caching, use all the server cores and autoscaling the number of threads on demand.
All the configs are optimized to work taking into account all the Kolibri features.

If you want to install it you need to use this ppa https://launchpad.net/~learningequality/+archive/ubuntu/kolibri-proposed/
If you want to take a look at its sources, they are available at https://github.com/learningequality/kolibri-server .

It has been tested in a bunch of different setups, from Raspberry Pi to huge servers. It’s not officially released yet because there are still some things to test but it’s already in use some real installations with actual learners with success.

José L.

7

THe server was indeed running 0.11 and the problems went away with an upgrade to 0.12.2. Sorry for the confusion. I am still going to run the one from the proposed repos though with nginx/uwsgi. I already had it running myself but I’d rather use a “supported” install (well, community support of course for a “proposed” version, but still). And if I find any issues there, I’ll report them.