[RESOLVED] Error syncing from local kolibri studio

Support Studio
1

We need to upload more than 200GB of content to offline kolibri servers
In order to achieve this goal, we have installed locally a Kolibri Studio and a kolibri-server in the same physical server (ubuntu 18.04) and tried to sync both
For Kolibri Studio we followed the instructions published in github until the “Running the development server” section

For kolibri-server we just change the redis port to avoid conflicts with the redis in docker (used by Studio) and we also changed the URLs in options.ini pointing to 0.0.0.0:8080 (Studio url)
Both services are working. In Studio we are able to create channels, upload files, publish channels.
When we tried to import channels from kolibri-server it immediately detects the local Studio, but when it tries to sync any content then we get an error.
In the terminal we noticed that when kolibri tries to sync the content, Django send messages regarding that /content/databases/[channel ID] sqlite3 was not found
Is there any step that we are missing?
Thanks!

2

hello Juan_Pablo, not sure what can be the reason and we usually don’t provide support on Studio because it’s not a product thought to be installed by normal users.
In any case, I’d check the logs in the Studio server to see what kind of error it shows when it should provide those files. It smells to some problem in the celery queues, but it’s hard to confirm it anyway without having access to the server (one more reason to avoid using Studio as a ready-to-go product, it’s not and requires an important engineering effort to deploy it)

Regards
José

3

Hi Jose, thanks for your prompt response. I agree with you, we would not like to deal with a local Studio, but unfortunately it is the only way we have to upload a huge amount of content to kolibri servers.
Here is a copy of the django.log file
https://docs.google.com/document/d/1ns42Ph8a1qpri1HW3DAa5VOpzEVpyz0RdJh4NFqUMYs/edit?usp=sharing

I have reinstalled everything and have enabled a fresh image of the server through Anydesk
kolibri server = localhost:8008
kolibri admin = admin
kolibri password = 123lkj

For your reference here is step by step of what we have done to install Studio
https://docs.google.com/document/d/1jl6BU-8Ohq_dBgOEch-BB3H6Q17t-RorQf6JQRh8q1E/edit?usp=sharing

Hope you may take a short look and give us some clues how to solved the error. We really appreciate your cooperation.
Thank you very much!

4

Hi @Juan_Pablo_Seminario,

Running Studio as a local server requires some routing for resources in storage, such as the channel databases and uploaded files. In production, requests are routed directly to our storage instance outside of the app code.

Here’s a diff of changes that should get it working:

diff --git a/contentcuration/contentcuration/dev_urls.py b/contentcuration/contentcuration/dev_urls.py
index ec4369fd1..bf4afda20 100644
--- a/contentcuration/contentcuration/dev_urls.py
+++ b/contentcuration/contentcuration/dev_urls.py
@@ -1,12 +1,16 @@
+import urllib.parse
+
 from django.conf import settings
 from django.contrib import admin
 from django.http.response import HttpResponseRedirect
+from django.http.response import HttpResponseNotFound
 from django.urls import include
 from django.urls import path
 from django.urls import re_path
 from drf_yasg import openapi
 from drf_yasg.views import get_schema_view
 from rest_framework import permissions
+from django.core.files.storage import default_storage
 
 from .urls import urlpatterns
 
@@ -19,6 +23,25 @@ def webpack_redirect_view(request):
     )
 
 
+def file_server(request, storage_path=None):
+    # generate the minio storage URL, so we can get the GET parameters that give everyone
+    # access even if they don't need to log in
+    if storage_path is None:
+        return HttpResponseNotFound()
+
+    params = urllib.parse.urlparse(default_storage.url(storage_path)).query
+    host = request.META['HTTP_HOST'].split(":")[0]
+    port = 9000  # hardcoded to the default minio IP address
+    url = "http://{host}:{port}/{bucket}/{path}?{params}".format(
+        host=host,
+        port=port,
+        bucket=settings.AWS_S3_BUCKET_NAME,
+        path=storage_path,
+        params=params,
+    )
+    return HttpResponseRedirect(url)
+
+
 schema_view = get_schema_view(
     openapi.Info(
         title="Kolibri Studio API",
@@ -47,6 +70,7 @@ urlpatterns = urlpatterns + [
         r"^redoc/$", schema_view.with_ui("redoc", cache_timeout=0), name="schema-redoc"
     ),
     re_path(r"^api-auth/", include("rest_framework.urls", namespace="rest_framework")),
+    re_path(r"^content/(?P<storage_path>.+)$", file_server),
 ]
 
 if getattr(settings, "DEBUG_PANEL_ACTIVE", False):

Regards,
Blaine

1 Like