Optimisation of mp4 videos

Support Kolibri
21

Hi @richard
A question please:

If I install the Kolibri RPi image, the system will be running kolibri-server. Correct?

Assuming this is correct, is it then possible to downgrade to kolibri?

If not, then it seems that to be able to use kolibri, we need to install RPiOS and then install kolibri from the PPA as described in the Raspberry Pi section here: Install Kolibri — Kolibri 0.11.0.dev documentation

Then we have to manually configure the system to get it to look like the Kolibri RPi image installation eg AP setup etc.

Is that correct, or is there a way that we can install from an RPi image and get kolibri rather than kolibri-server?

Thanks

22

Hi @jredrejo
I tried swapping between kolibri-server and kolibri on the Kolibri RPi image installation but without success.

So I set up two separate devices, one with the image installation running kolibri-server and the other with kolibri installed on RPiOS (Debioan 11 Bullseye) and ran the comparison tests. Both are Kolibri Ver 0.15.5

The basic test initially is just to launch 24 concurrent video sessions using the KA (US) channel Entrepreneurship videos which have been optimised for Quickstart.
Both systems have WiFi disabled for testing.

There is a very marked difference in the performance between the two devices.

The kolibri-server installation hits a Load Average figure of 7 to 8 during the video launch phase and starts to become unresponsive.

By comparison, the kolibri installation does not exceed Load Average figure of 1 and remains quite responsive.

For our usage scenario, I think we will have to use the kolibri based installation.
Our goal is to have one Kolibri RPi server connected to 5 classroom APs.
Adding the Kolibri RPi server doubles the cost of a 5 classroom installation.
Our classroom current classroom APs serve 100GB of static content and have proved very reliable over a number of years. We have installations that have run without a technical site visit for 5 years.

What we are seeking to gain from adding the Kolibri server is the Learning Platform capabilities ie access to a flexible on-line content repository, student exercises and tests, and tracking of student progress.
All of this looks quite possible on the kolibri installation at the scale we envisage.

What we are very wary of is losing the system reliability as most sites have no local technical support available.

The whole Learning Platform concept is very new to teaching staff at most sites and it will take some time just to get the basics in place and working properly.

Moving to the kolibri-server installation appears to limit the scalability on the RPi platform.
The additional features of kolibri-server (ie multi-thread support) seem to be of less value than the loss of scalability in our deployment scenario.

My impression from our discussions is that kolibri-server will not add significant performance improvement for serving simple static content like videos, nor for the student exercise/test pages and student progress tracking.

It is used for unpacking the PHET zip files on the fly, but these are fairly small files anyway, so the performance hit using kolibri is probably minimal. Are there other Channels that use zip files?

The Coaching capabilities are not on the agenda for the teaching staff at this stage, so we won’t notice any difference there.

Is this a reasonable assessment?

Regards
Terry

23

hi @tgillett kolibri-server is nothing but a set of configs added to run kolibri with nginx/uwgsi/redis.
Installing kolibri-server always installs the latest kolibri version. You don’t need to install it.

In the Kolibri RPi image, you can uninstall the package and nginx and kolibri will remain to be setup/used as usual.

24

Hi @jredrejo
Thanks for that explanation.

So just to make sure I understand correctly, if I install with the Kolibri RPi image, and then I want to switch from from ‘kolibri-server’ to just run ‘kolibri’ all I have to do is remove kolibri-server package with
$ sudo apt-get -y remove kolibri-server

Then I guess a restart would be good just to be sure everything is co-ordinated.

Is that correct? (very convenient if that is the case).

Regards
Terry

25

Hello @tgillett , you mention several things in your comment, I’ll try to response all.

  • I still would like to know why kolibri-server works worse for you than just kolibri. kolibri-server adds configs over kolibri, it’s not a different version. I haven’t had a chance to try to reproduce your tests yet but I’m very interested in knowing what could be causing the difference.
  • You said you couldn’t swap between them, can I know why? They are just systemd services that can be enabled/disabled, if you can share the problem you found we could try to solve it and you could do a real test with the same OS and just checking between a simple kolibri or the kolibri-server configs over it.
  • What you said is part right: kolibri-server will not add significant performance improvement for serving simple static content like videos , but
  • this is incorrect. nor for the student exercise/test pages and student progress tracking. kolibri-server should improve this part, and any of the parts of kolibri that involve cpu processing.
  • About the zip files: All channels having html5 applications are zip files.
  • Coaching capabilities are one of the most cpu intensive parts of kolibri, so that’s one less reason for you to use kolibri-server.

I hope to have solved all your doubts, but let me insist: it would be very interesting for us and our users if you could test swapping in the same installation between kolibri-server and kolibri.
These are the instructions I gave you yesterday to stop kolibri-server:

sudo systemctl stop kolibri-server
sudo systemctl stop nginx

then, to start just kolibri you can try adding the foreground option:
kolibri start --foreground
this will provide debugging info in the console so we can find out why you can’t start it

Regards
José

26

The problem was my misunderstanding of how to changeover.

I will now try it with your suggested approach (and a better understanding of how the two relate!).

Thanks again for all your help. Hopeful this time I will get some better info for you.

Regards
Terry

27

Hi @jredrejo

I tried your suggestion for shutting down kolibri-server and running kolibri but it did not go well.
This is on a clean installation that was running ok.

Have I missed something?

Here is the sequence of events:

pi@kolibri:~ $ sudo systemctl stop kolibri-server
pi@kolibri:~ $ sudo systemctl stop nginx
pi@kolibri:~ $ kolibri start --foreground
INFO Option DEBUG in section [Server] being overridden by environment variable KOLIBRI_DEBUG
INFO Option DEBUG_LOG_DATABASE in section [Server] being overridden by environment variable KOLIBRI_DEBUG_LOG_DATABASE
INFO Running Kolibri with the following settings: kolibri.deployment.default.settings.base
INFO Importing ‘tasks’ module from django apps
INFO Starting Kolibri 0.15.5
INFO Bus state: ENTER
INFO Listening for SIGTERM.
INFO Listening for SIGHUP.
INFO Listening for SIGUSR1.
INFO Listening for SIGINT.
INFO Bus state: IDLE
INFO Bus state: START
INFO Starting async task workers.
INFO Started monitor thread ‘ZeroConfPlugin’.
INFO Started monitor thread ‘ProcessControlPlugin’.
INFO Starting on 0.0.0.0:81
ERROR Error in HTTP server: shutting down
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/plugins/servers.py”, line 177, in _start_http_thread
self.httpserver.start()
File “/usr/lib/python3/dist-packages/kolibri/dist/cheroot/server.py”, line 1836, in start
self.prepare()
File “/usr/lib/python3/dist-packages/kolibri/dist/cheroot/server.py”, line 1791, in prepare
raise socket.error(msg)
OSError: No socket could be created – ((‘0.0.0.0’, 81): [Errno 13] Permission denied)

INFO Bus state: STOP
INFO HTTP Server kolibri.utils.server.Server((‘0.0.0.0’, 80)) already shut down
INFO HTTP Server kolibri.utils.server.Server((‘0.0.0.0’, 81)) already shut down
INFO Joining ‘ProcessControlPlugin’
ERROR Error in ‘START’ listener <bound method ZipContentServerPlugin.START of <kolibri.utils.server.ZipContentServerPlugin object at 0xb3da48f0>>
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/base.py”, line 273, in publish
result = listener(*args, **kwargs)
File “/usr/lib/python3/dist-packages/kolibri/utils/server.py”, line 187, in START
super(ZipContentServerPlugin, self).START()
File “/usr/lib/python3/dist-packages/kolibri/utils/server.py”, line 156, in START
super(ServerPlugin, self).START()
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/plugins/servers.py”, line 164, in START
self.wait()
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/plugins/servers.py”, line 198, in wait
raise self.interrupt
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/plugins/servers.py”, line 177, in _start_http_thread
self.httpserver.start()
File “/usr/lib/python3/dist-packages/kolibri/dist/cheroot/server.py”, line 1836, in start
self.prepare()
File “/usr/lib/python3/dist-packages/kolibri/dist/cheroot/server.py”, line 1791, in prepare
raise socket.error(msg)
OSError: No socket could be created – ((‘0.0.0.0’, 81): [Errno 13] Permission denied)

INFO Starting on 0.0.0.0:80
ERROR Error in HTTP server: shutting down
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/plugins/servers.py”, line 177, in _start_http_thread
self.httpserver.start()
File “/usr/lib/python3/dist-packages/kolibri/dist/cheroot/server.py”, line 1836, in start
self.prepare()
File “/usr/lib/python3/dist-packages/kolibri/dist/cheroot/server.py”, line 1791, in prepare
raise socket.error(msg)
OSError: No socket could be created – ((‘0.0.0.0’, 80): [Errno 13] Permission denied)

INFO Bus state: IDLE
INFO Bus state: EXIT
INFO Waiting for child threads to terminate…
INFO Waiting for thread JOBCHECKER-cd0014355c46443aaa3d86c8c950cd08.
ERROR Error in ‘START’ listener <bound method KolibriServerPlugin.START of <kolibri.utils.server.KolibriServerPlugin object at 0xb539ec70>>
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/base.py”, line 273, in publish
result = listener(*args, **kwargs)
File “/usr/lib/python3/dist-packages/kolibri/utils/server.py”, line 175, in START
super(KolibriServerPlugin, self).START()
File “/usr/lib/python3/dist-packages/kolibri/utils/server.py”, line 156, in START
super(ServerPlugin, self).START()
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/plugins/servers.py”, line 164, in START
self.wait()
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/plugins/servers.py”, line 198, in wait
raise self.interrupt
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/plugins/servers.py”, line 177, in _start_http_thread
self.httpserver.start()
File “/usr/lib/python3/dist-packages/kolibri/dist/cheroot/server.py”, line 1836, in start
self.prepare()
File “/usr/lib/python3/dist-packages/kolibri/dist/cheroot/server.py”, line 1791, in prepare
raise socket.error(msg)
OSError: No socket could be created – ((‘0.0.0.0’, 80): [Errno 13] Permission denied)

INFO Bus state: START_ERROR
ERROR Exiting due to error in start listener:
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/base.py”, line 215, in _transition
return self.publish(newstate, *args, **kwargs)
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/base.py”, line 291, in publish
raise exc
magicbus.base.ChannelFailures: OSError(“No socket could be created – ((‘0.0.0.0’, 81): [Errno 13] Permission denied)”)
OSError(“No socket could be created – ((‘0.0.0.0’, 80): [Errno 13] Permission denied)”)

INFO Bus state: STOP
INFO HTTP Server kolibri.utils.server.Server((‘0.0.0.0’, 80)) already shut down
INFO HTTP Server kolibri.utils.server.Server((‘0.0.0.0’, 81)) already shut down
INFO Joining ‘ProcessControlPlugin’
INFO Stopped thread ‘ProcessControlPlugin’.
INFO Stopped thread ‘ProcessControlPlugin’.
INFO Joining ‘ZeroConfPlugin’
INFO Joining ‘ZeroConfPlugin’
INFO Stopped thread ‘ZeroConfPlugin’.
INFO Stopped thread ‘ZeroConfPlugin’.
INFO Asking job schedulers to shut down.
INFO Asking job schedulers to shut down.
INFO Waiting for thread HTTPServer Thread-3.
INFO Bus state: IDLE
INFO Bus state: EXIT
INFO Bus state: EXITED
INFO Waiting for child threads to terminate…
INFO Waiting for thread HTTPServer Thread-3.
Exception in thread HTTPServer Thread-3:
Traceback (most recent call last):
File “/usr/lib/python3.7/threading.py”, line 917, in _bootstrap_inner
self.run()
File “/usr/lib/python3.7/threading.py”, line 865, in run
self._target(*self._args, **self._kwargs)
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/plugins/servers.py”, line 177, in _start_http_thread
self.httpserver.start()
File “/usr/lib/python3/dist-packages/kolibri/dist/cheroot/server.py”, line 1836, in start
self.prepare()
File “/usr/lib/python3/dist-packages/kolibri/dist/cheroot/server.py”, line 1791, in prepare
raise socket.error(msg)
OSError: No socket could be created – ((‘0.0.0.0’, 81): [Errno 13] Permission denied)

Exception in thread HTTPServer Thread-4:
Traceback (most recent call last):
File “/usr/lib/python3.7/threading.py”, line 917, in _bootstrap_inner
self.run()
File “/usr/lib/python3.7/threading.py”, line 865, in run
self._target(*self._args, **self._kwargs)
File “/usr/lib/python3/dist-packages/kolibri/dist/magicbus/plugins/servers.py”, line 177, in _start_http_thread
self.httpserver.start()
File “/usr/lib/python3/dist-packages/kolibri/dist/cheroot/server.py”, line 1836, in start
self.prepare()
File “/usr/lib/python3/dist-packages/kolibri/dist/cheroot/server.py”, line 1791, in prepare
raise socket.error(msg)
OSError: No socket could be created – ((‘0.0.0.0’, 80): [Errno 13] Permission denied)
INFO Waiting for thread HTTPServer Thread-4.

pi@kolibri:~ $ systemctl status kolibri
● kolibri.service - LSB: kolibri daemon, an offline education platform
Loaded: loaded (/etc/init.d/kolibri; disabled; vendor preset: enabled)
Active: inactive (dead)
pi@kolibri:~ $

28

yes, the problem is clear: when you configured kolibri-server you said it to be in the tcp port 80. Kolibri is trying to start in the same port and that’s not possible for an user without permissions:

OSError: No socket could be created – ((‘0.0.0.0’, 80): [Errno 13] Permission denied)

The easiest fix is to edit /home/pi/.kolibri/options.ini, look for this section:

[Deployment]
URL_PATH_PREFIX = /
HTTP_PORT = 80
ZIP_CONTENT_PORT = 81

and change those ports to 8080, 8081
then you’ll be able to start kolibri as a pi user and it will run in those ports.

Another option, instead of editing the options.ini configuration file, is executing:
sudo dpkg-reconfigure kolibri-server and set it to a port 8080 (ports below 1000 are not authorized for normal users)

After changing these ports you should be able to run
kolibri start --foreground
without problems.

José

29

Many thanks, I will give that a try and let you know how it goes.

30

OK That worked perfectly!

After switching to kolibri, I did the initial test of launching 24 videos.
The result was fine, all videos launched correctly and the system remained responsive.
The Load Average peaked at around 0.8 and the bounced between 0.25 and 0.75

What should I test next to give you some worthwhile feedback?

31

It would be great to have a comparison between:

  • Kolibri RPI image as it is (running kolibri-server)
  • Same image stopping kolibri-server and running just kolibri
  • Your sistem with the kolibri package installed (I understand your system is a Raspbian so, theorically the only difference with the Kolibri RPI image should be redis, hostapd and dnsmasq)

But if I understand your comments correctly, you get aprox the same results whenever kolibri runs alone and a worse performance running kolibri-server, right? (performance in terms of number of clients connected + average cpu load). The experience with previous versions of kolibri has always been that many more connections could be done using kolibri-server than using kolibri alone.

If your results are correct it’s clear we should investigate if either uwsgi or nginx are causing some bottleneck (honestly, with the latest changes in caching and serving with kolibri 0.15.5 nginx might be removed from kolibri-server in the short term).
Thanks
José

32

OK I will run through the list tomorrow and get back to you.
Thanks

33

Just a quick clarification about the number of connection you mentioned.

If I run 4 client devices, each running 6 different videos in browser tabs, does that count as 24 connections?

34

Hi @jredrejo

As you suggested, I ran a series of video launching tests using the Kolibri RPi image in both kolibri-server and kolibri modes, plus a test of RPiOS plus Kolibri installation for comparison. These were run on a RPi 4B with 4GB RAM.
I also ran a test of the Kolibri RPi image installation in kolibri-server mode on a RPi 4B with 2GB RAM to see if limited RAM would contribute to the issue.

In summary…

  • The tests of the kolibri-server mode installations all showed high Load Average values and failed to launch some videos.
  • The tests of kolibri mode installations showed low Load Average values and launched all videos successfully.
  • The test of kolibri-server mode installation on the 2GB RAM board showed higher Load Average values than the 4GB RAM board, but otherwise performed similarly.

Details of the tests are below.

Please let me know your thoughts.

Common test conditions:
Laptop clients are i5 Ubuntu 22.04 connected via WiFi to an Access Point connected to the RPi by Ethernet.
Web Browser on laptop clients is Firefox.
Each laptop is logged in to Kolibri as a different student account.
Videos are the Entrepreneurship videos from KA (US) optimised for Faststart.
Load Average monitored on RPi with ‘top -d2’ command.

Test process:
Open the Channel page at the Entrepreneurship page on each laptop.
Launch the 6 videos in each browser by ‘Open in a new tab’ on each laptop.
Allow 5 seconds between each launch.
After launching all tabs on all laptops, step through all the tabs and start each video playing.

Test1.
Kolibri RPi image 0.15.5 on RPi 4B 4GB RAM
Two laptop clients connected. (2x6 test)

Result: Load Average rises to 3.0 during launch, dropping back to 1.0 after 5 mins.
All videos launch and run ok.

Test 2.
Kolibri RPi image 0.15.5 on RPi 4B 4GB RAM
Four laptop clients connected. (4x6 test)

Result: Load Average rises to 5.5 during launch, dropping back to 1.0 after 5 mins.
One video tab failed to launch and run ok.

Test 3.
Kolibri RPi image 0.15.5 on RPi 4B 4GB RAM
Six laptop clients connected. (6x6 test)

Result: Load Average rises to 7.5 during launch, dropping back to 2.0 after 5 mins.
Three video tabs failed to launch and run ok.

Test 4.
Kolibri RPi image 0.15.5 on RPi 4B 4GB RAM. Reset to run kolibri rather than kolibri-server.
Six laptop clients connected. (6x6 test)

Result: Load Average rises to 0.7 during launch, dropping back to 0.65 after 5 mins.
All video tabs launched and ran ok.

Test 5.
RPiOS plus Kolibri 0.15.5 on RPi 4B 4GB RAM
Six laptop clients connected. (6x6 test)

Result: Load Average rises to 0.48 during launch, dropping back to 0.4 after 5 mins.
All video tabs launched and ran ok.

Test 6.
RPiOS plus Kolibri 0.15.5 on RPi 4B 2GB RAM
Six laptop clients connected. (6x6 test)

Result: Load Average rises to 14.0 during launch, dropping back to 3.0 after 5 mins.
Four video tabs failed to launch and run ok.

35

Correction:

Test 6.
Kolibri RPi image 0.15.5 on RPi 4B 2GB RAM.

36

Hi @jredrejo

In order to move forward with our field trials, I propose to use install the Kolibri RPi image and switch to ‘kolibri’ in lieu of ‘kolibri-server’ as follows:

$ sudo systemctl disable nginx
$ sudo systemctl disable kolibri-server
$ sudo systemctl enable kolibri
$ sudo nano /home/pi/.kolibri/options.ini # Set the ports to 8080 and 8081
$ sudo shutdown -r now

I have tested this on the bench and all seems to be well.
Do you see any problems with this approach?

Thanks
Terry

37

I don’t see any problem, you can even add a latets step:apt-get remove kolibri-server if you’re not using it.

In any case, I’ll build a table with your prior results to try to see the differences and will do our own tests. Your results don’t make sense from a technical point of view: using 4 cores should provide you more performance than using just one, so there must be some key config that’s causing problems after the latest kolibri release and I want to find it out to fix it.
In previous versions of kolibri the results were quite clear in favour of using kolibri-server (which at the end it’s nothing else but adding uwsgi, redis and a nginx proxy)
Any idea you can have of what might cause the difference is very welcome

Regards
José

38

Thanks for the feedback.

I have left the kolibri-server module in place as we can switch back to it quite easily for testing if we need to.

Can you point me to a previous version of the RPi image that you have tested ok and I will run the same tests to compare?

One thing I would like to get back to is to be able to use port 80 rather than 8080, so the standard RPi image with the kolibri-server option is a simpler installation target.

39

HI @jredrejo

Just wondering if you have any progress update on the RPi performance differences for ‘kolibri-server’ and ‘kolibri’ modes?

Also whether you have posted any optimised mp4 files in the online channel library that we could download and test?

Meanwhile we have sent out some trial units set up as I described previously with all the mp4 files optimised locally and with the RPi running in ‘kolibri’ mode. Hopefully we will start to get some real world results back over the next few weeks.

Regards
Terry