Notes on building the cordova-based Moodle app for Android

A collection of the various issues faced and workarounds - building older and newer versions of the Moodle Android app. Many of these might be applicable to other cordova apps too. This post encapsulates work done, trial and error, over a one-month period.

Edit: Building v4.0.2 is described in a newer post.
Later Edit: Building v4.1.0 - release apk in Github Actions - is described in a newer post.

• Build notes on successfully building v4.0.1 - https://hnsws.blogspot.com/2022/09/modify-and-build-moodle-app-version-4.html
  
  
• Unfortunately, v400 branch of the moodle app has some really nasty bugs like https://tracker.moodle.org/browse/MOBILE-4135
  
  
• Incomplete Travis yml for building v4.0.1 - https://github.com/hn-88/moodleapptravistest/blob/937a2842c351c1cf13fe8270d30ed4521d481376/.travis.yml - there needs to be
  npm run build:prod
  before
  npm run prod:androidnorun
  
  
• As seen in the above yml, though the travis build image only has SDK 29 pre-installed, we can install SDK 30 and the relevant build tools using sdkmanager,
  yes | sdkmanager "build-tools;30.0.3"
  
  
• Travis builds were relatively easier to set up since the repo already had the appropriate build image set up in the included travis yml. Appveyor build did not work, probably because of not setting up the environment properly. Probably can be done with commands like sdkmanager above for SDK 31 and above, since they use openjdk and don't depend on Oracle's java.
  
  
• Travis yml for building v3.9.5 - https://github.com/hn-88/moodleappAppveyorTrial/blob/44a5c27212559c18b78f70d12fff952c6ed0c8fd/.travis.yml - but the resulting app doesn't work, I think, maybe due to ionic/android sdk incompatibilities.
  
  
• For the above travis yml, commands like
  npm run build:prod
  use scripts which are in package.json scripts section. We can instead use commands like
  npx ionic build --prod
  directly in the yml, so that we don't have to edit package.json. Another caution is that the options needed for the build command may vary for different versions of cordova. So we need to check the documentation or rely on the existing commands in the moodleapp repo. Some links on command-line build:
  
  https://stackoverflow.com/questions/10930331/how-to-sign-an-already-compiled-apk
  or
  https://stackoverflow.com/questions/26449512/how-to-create-a-signed-apk-file-using-cordova-command-line-interface
  make a build conf and then ...
  
  
  (or import android project to windows and build - that does not work out of the box - there are some paths which have / instead of \ or vice-versa, and some amount of path mangling has to be done.
  https://stackoverflow.com/questions/44959885/cordova-to-android-studio-error-malformed-uxxxx-encoding)
  
  

  Examples at

  Building an Android Project - Travis CI (travis-ci.com)
  

  Calc/.travis.yml at master · samlsso/Calc (github.com)
  
  
  
• Build notes on successfully building v3.9.4 - https://hnsws.blogspot.com/2021/04/customizing-and-building-moodle-app.html
  
  
• The zip path traversal vulnerability in v3.9.4 can probably be fixed easily using https://tracker.moodle.org/browse/MOBILE-3949
  which points to the diff at
  https://github.com/moodlehq/moodleapp/pull/3024/files
  
  
• For older versions of the app, we need to specify an older version of cordova-res, since the latest version of cordova-res has dependencies which need npm>12
  
  npm i -g cordova-res
  /home/homedir/.nvm/versions/node/v11.15.0/bin/cordova-res -> /home/homedir/.nvm/versions/node/v11.15.0/lib/node_modules/cordova-res/bin/cordova-res
  
  > sharp@0.29.3 install /home/homedir/.nvm/versions/node/v11.15.0/lib/node_modules/cordova-res/node_modules/sharp
  > (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
  
  sharp: Installation error: Expected Node.js version >=12.13.0 but found 11.15.0
  sharp: Please see https://sharp.pixelplumbing.com/install for required dependencies
  
  
  
  from older versions at
  https://www.npmjs.com/package/cordova-res
  
  
  
  npm i -g cordova-res@0.15.1
  
  worked.
  
  
  
• Probably the travis yml above (for 3.9.5) will work for building 3.9.4, but we would need to use SDK 29, because
  
  
• Though we can change the SDK version in Android Studio 4.1 and build v3.9.4 using targetSdkversion=30, the resulting app has the problem, "Could not read database" error when trying to log in to Moodle. If we use SDK 29, this issue is not seen. Probably due to this - https://github.com/andpor/react-native-sqlite-storage/issues/462
  
  
• In any case, Google play store mandates target SDK 31 from November, and Moodle app builds targeting SDK 31 just will not work. As mentioned in https://stackoverflow.com/questions/68344424/unrecognized-attribute-name-module-class-com-sun-tools-javac-util-sharednametab,
   need to change jdk to 11 for sdk 31 and above. Then the cordova build fails for cordova<11. And older versions of the Moodle app are incompatible with higher versions of npm which are needed for cordova 11. How to target SDK 30 is mentioned at https://github.com/apache/cordova-android/issues/1178#issuecomment-814116954 - but those changes were not enough in my case. I first tried
  grep -rn . -e "targetSdk"
  via https://stackoverflow.com/questions/16956810/how-to-find-all-files-containing-specific-text-string-on-linux
  and then used Android Studio's find / replace in files to change targetSdkversion in a couple of instances of files not mentioned above, so files to modify were
  /platforms/android/project.properties
  /platforms/android/CordovaLib/project.properties
  /platforms/android/app/src/main/res/xml/config.xml
  as well as
  /platforms/android/build.gradle
  /platforms/android/gradle.properties
  (which had the cdvTargetSdkVersion=29 line)
  
  
• For changing version code of the app - Google Play store insists on incrementing version code - we need to change it in src/config.json for v3.x and config.xml for v4.x and then doing the whole npx ionic cordova remove android, then add, etc. Other methods, like changing flavors in Android Studio etc, don't work properly, with the version inside the app showing an older version if we don't do it this way. That makes sense, since the ionic build is what creates the webpages which are seen as screens in the app.
  
  
• If we try build tools 31 or later, older versions of Android Studio will complain that the build tools are corrupted. https://stackoverflow.com/questions/68387270/android-studio-error-installed-build-tools-revision-31-0-0-is-corrupted
  If we just try copying d8 to dx as mentioned by some of the people in the above post, that doesn't help with fixing all the incompatibilities. Easier to just build with compatible versions of cordova / gradle / android studio. That is, old code --> old cordova --> old build tools + old android studio.
• https://developer.android.com/studio/releases/build-tools
  mentions how we should target a particular build tools version.
  
  
  "To use a different version of the build tools, specify it using buildToolsVersion in your module's build.gradle, as follows:
   
  
  android {
      buildToolsVersion "30.0.2"
      ...
  }"
  
  
  But instead, what I was doing was, if it complained that build tools 31 is corrupted, I would just unselect it from the SDK manager window - uninstall it. Then, by default, Android Studio would use the next highest available build tools version to build. 
  
  
• We cannot build v3.9.4 with later versions of Android Studio - version 4.2, 2021.3 and so on. Probably because they use later versions of gradle which are incompatible with cordova 9. I believe gradle version 6.x is required for compiling the app without errors, while later Android Studio versions need versions 7 and above.
  
  
• https://stackoverflow.com/questions/32258183/how-to-install-an-older-version-of-android-studio
  https://developer.android.com/studio/archive
  The links to 4.2.x and 4.1.x tar.gz files were working when this was written.
  
  
• Changing the gradle versions in a project in Android Studio can be done via various methods as described at https://www.geeksforgeeks.org/how-to-update-gradle-in-android-studio/
  
  
• Changing the desired gradle and android gradle plugin versions in Android Studio is via File --> Project Structure --> Project
  
  
• While testing builds, it takes around 90 seconds for the moodle app to be available in the Chromium browser after issuing npm start - does ionic compile etc.
  
  
• If we want to build entirely from the command-line, without using Android Studio, that also should be possible if the correct versions of gradle and SDK tools are installed. Simply trying
  sudo apt install gradle
  npm run dev:android
  said "Run successful" for v400 app (cordova 10) on Linux Mint 21 (based on Ubuntu 22.04). But there may be incompatibilities with gradle versions. sdkman can help.
  
  error:
  Unrecognized option: --illegal-access=permit
  Error: Could not create the Java Virtual Machine.
  Error: A fatal exception has occurred. Program will exit.
  
  gradle -version
  
  ------------------------------------------------------------
  Gradle 4.4.1
  
  https://stackoverflow.com/questions/68565856/unrecognized-option-illegal-access-permit-when-running-cordova-build-android
  
  
  
  https://gradle.org/install/
  
  
  curl -s "https://get.sdkman.io" | bash
  
  source "/home/myhomedir/.sdkman/bin/sdkman-init.sh"
  
  sdk install gradle 7.5.1
  
  
• After gradle errors due to incompatible versions, just uninstalling gradle with
  sudo apt remove gradle
  was not sufficient. Android Studio still would not build the project. Found that we have to remove both hidden folders
  ~/.gradle and projectfolder/.gradle
  for all the crud to be removed so that Android Studio would commence to build again properly.
  
  
• Documentation about signing the apk built using cordova - 
  
  https://cordova.apache.org/docs/en/latest/guide/platforms/android/index.html#extending-buildgradle
  

  
  

  https://stackoverflow.com/questions/26449512/how-to-create-a-signed-apk-file-using-cordova-command-line-interface
  

  
  

  maybe just give the --keystore & pw? as variables?
  
  
• If signing key is not provided and a release apk is built, we can sign using apksigner or jarsigner -
  
  android - How to Sign an Already Compiled Apk - Stack Overflow
  or use this GUI tool to avoid typing out all the paths -
  
  Android APK Signer & Aligner - Luke Alderton
  
  
• https://docs.travis-ci.com/user/languages/android/#How-to-install-Android-SDK-components
  is outdated.
  
  
  platform tools 29.0.3 is what is being used, setting target api 30 seems to work.
  
  
  unfortunately, still not signed. (Probably due to incompatible command-line options in the relevant cordova version.)
  
  
  
  Also tried uber-apk-signer
  https://stackoverflow.com/questions/10930331/how-to-sign-an-already-compiled-apk - but that did not work for some reason. 
  
  

modify and build the Moodle app version 4+

Edit - There is a newer post on building the release version entirely from Github Actions - for the moodle app v4.1.0.

There are a few changes from my earlier post about building the Moodle app. I'll try to write this post as a bash script which could be run on an Ubuntu-based machine which does not have the build environment set up, so that it would be possible to use something like Travis CI to build it. The current script below will not work without a gui on headless servers, since there are some steps like previewing in Chromium, installing android studio etc require a GUI. 

# the following are for a headless server, to get a gui over VNC

# https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-vnc-on-ubuntu-18-04

sudo apt update

sudo apt install xfce4 xfce4-goodies -y

sudo apt install tightvncserver -y

vncserver

# This starts up VNC server and asks to set up a password.

# The following will be after logging on to VNC - my preferred method is 

# to tunnel port 5901 over ssh.

# following the documentation at https://moodledev.io/general/app/development/setup

sudo apt install git chromium-browser -y

# from https://github.com/nvm-sh/nvm#installing-and-updating

wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash

source ~/.bashrc

git clone https://github.com/moodlehq/moodleapp.git
cd moodleapp
git checkout v4.0.1

# Here we make all the customizations, described at the bottom of this post

cd ..

git clone https://github.com/ourPrivate/Repo-with-customization-files.git

cd  customizedMoodleApp4changes

cp -Rvf * ../moodleapp

cd ../moodleapp

 

# node version should be < v15, 

# https://github.com/moodlehq/moodleapp/blob/master/package.json#L178

nvm install 14.15.0

npm install

npm start 

# npm start is to see if it works in chromium browser 

# Initially it will open the browser and complain connection refused.

# It will take around 5 minutes to compile, only then a browser reload will work.

# Even after the first compile, it takes around 90 seconds on a 16 GB machine for

# each npm start command - till the chokidar error comes up.

# Fix the chokidar error - too many watch files - with

# https://stackoverflow.com/questions/55763428/react-native-error-enospc-system-limit-for-number-of-file-watchers-reached

# echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p 

# Check if this is OK with

# cat /proc/sys/fs/inotify/max_user_watches 

 

# For building Android or iOS app, need more setup,

# https://moodledev.io/general/app/development/setup

# via my previous post, need to install cordova - 

npm install cordova

npm i -g cordova-res

sudo apt install libsecret-1-dev -y

sudo apt install openjdk-11-jdk -y

sudo add-apt-repository ppa:maarten-fonville/android-studio

sudo apt update

sudo apt install android-studio -y

# This ppa install does not add the required environment variables, so

# to find java home, via https://www.baeldung.com/find-java-home

dirname $(dirname $(readlink -f $(which javac)))

export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64

export PATH=$JAVA_HOME/bin:$PATH

export ANDROID_SDK_ROOT=/home/azureuser/Android/Sdk

export PATH=$ANDROID_SDK_ROOT/tools:$ANDROID_SDK_ROOT/tools/bin:$ANDROID_SDK_ROOT/platform-tools:$PATH 

nano .bashrc

# and added all these export commands to the end of .bashrc also

# We may need to install the android-30 SDK separately,

# https://developer.android.com/studio/command-line/sdkmanager

# but it is far easier to do it from within the gui SDK manager

npm run prod:android 

# This will probably fail due to not finding gradle, 

# and at this point, we can just open android studio, 

# import the platforms/android folder, and build there.

# The android studio build may complain that build tools ver 30 are not found.

# The way to install them using SDK manager is to choose

# the SDK Tools tab, and check the "Show package details"

# in order to make the Build tools 30.0.3 show up, which we can mark and install. 

 

# Unfortunately, building this way with the above steps seems to result in

# an app with the default cordova or ionic icons. To change to our icons, 

# we could use Android Studio's Image Asset Studio

# https://github.com/ionic-team/capacitor-assets/issues/108 

# But the standard way to generate icons as per

# https://capacitorjs.com/docs/guides/splash-screens-and-icons

# was creating files in android directory instead of platforms/android, so

# since mv cannot do it, 

#https://unix.stackexchange.com/questions/9899/how-to-move-and-overwrite-subdirectories-and-files-to-parent-directory 

cordova-res android --skip-config --copy 

cp -Rvf android/* platforms/android

# Then build with Android Studio. 

# https://stackoverflow.com/questions/37300811/android-studio-dev-kvm-device-permission-denied

# if you want to install an Android emulator

# sudo apt install qemu-kvm

#sudo adduser $USER kvm

# sudo reboot

# To run the emulator outside Android Studio in a separate window, 

# start the emulator before opening the project

# https://stackoverflow.com/questions/70986530/android-studio-emulator-in-a-separate-window 

To sign the app with our existing keystore, followed a procedure similar to my previous post, with a change that the location of the Build types tab has changed in the latest 2021.2.1 chipmunk version of Android Studio. In short, following the official documentation (which shows screenshots of the older version)

1. Project window --> right-click on app --> Open Module settings
   
   
   
   
   
2. Modules on LHS pane --> app --> Signing configs tab --> create a new signing config by giving it some name like releaseconfig
   
   
   
   
   
3. Modules --> app --> Build types tab --> choose the newly created signing config under "Signing" for the release build. (Extreme scrolling to the right was needed before the drop-down was visible, in one case)
   
   
   
   
   
4. Build variants --> choose the release build
   
   
   
   
   
   

The app build so far was fine. But unfortunately, the 4.00 branch of the Moodle app has show-stopping bugs - 

https://tracker.moodle.org/browse/MOBILE-4135

https://tracker.moodle.org/browse/MOBILE-4084

So, I'll continue trying fixes for these in separate posts. One option might be to use the 3.9.4 or 3.9.5 version and fix only the zip path traversal vulnerability in them - 

https://tracker.moodle.org/browse/MOBILE-3949

app for new Moodle instance - several options

There was a request for an app for a new Moodle instance. My response was, in part,

There are several options. 

1. Option within the same app - We could push out an update to our customized Moodle app which would enable users to choose between the two Moodle sites - this is fairly easy to implement.

2. Separate app - Having a separate app is certainly do-able. If we just change the name of the app, URL and (if necessary) the splash screen and icon, we could have a separate app. If you want that, please let me know the name of the app you wish to have, and if you want any changes to the splash screen and icon.

3. Web app - If you don't want the hassle of maintaining another app, we can use a technique like this,

https://medium.com/@sam20gh/how-to-add-add-to-home-screen-to-your-website-4b07aee02676

and have a qr code which students can scan, and then install the site as a web app to their home screen.

(Since the courses on this site are supposed to be interactive, some elements may not work on the mobile app, but might work on the web app. Comparison table is at

https://docs.moodle.org/403/en/images_en/2/2a/Moodle_features_students.pdf

The difference would be that the web app will save downloaded files outside the app, in the downloads folder.)

Edit - Option (1) was chosen.

moodle app build blues and fix

After my previous post about building the moodle app using Github actions, Moodle app version 4.1.1 would not build with the same commands. I guess there was some compatibility issues with the versions of cordova or gradle or SDK or .... 

Anyway, the newly released Moodle app version 4.2.0 builds without any issues with this basic workflow. I had seen an announcement that cordova-android is going to default to the latest version in future, so the workflow did not have to specify cordova-android@12.0.0.

Edit: 13 Jan - This commit forces a change to the moodle.config.json format. The Moodle tracker does not have any documentation about the expected format. By trial and error, found that this format works.

"sites": [

    {

      "name": "This seems to be ignored",

      "url": "https://our.moodle.instance.org"

    }

  ],

Customized Moodle app bug and solution

 Copy-pasting from a request for help in the Moodle forum - "we're facing a strange issue, and any help is welcome. The questions in the feedback module are visible when viewed on a web browser, but the questions are not visible when using a customized Moodle app. Even if we just build the Moodle app for Android without any customizations, the resulting app doesn't show the questions. "

I see now that this is a bug reported in the Bug Tracker for the app, https://tracker.moodle.org/browse/MOBILE-4698

The code in the "beta" branch has the fix, and the bug is fixed there, as I check today.

Google Play store blues - trying in vain to update our app - shortcut to update version code

Updates to one of our apps was being blocked by Google Play console.

Initially, in the developer console, there was a Policy Declaration we had to make, stating the reason for adding a particular sensitive permission - in this case, REQUEST_INSTALL_PACKAGES permission.

There was a form to appeal the rejection, but submitting that just gave a generic reply saying the appeal was rejected.

The Moodle app's repository meanwhile mentioned this issue, and gave a fix - https://tracker.moodle.org/projects/MOBILE/issues/MOBILE-4134?filter=doneissues

But even after uploading a version of the app with these fixes, the same generic automated rejection of the update. Then I tried the "reaching out to our policy support team" link at the bottom of the email. 

It was not very clear where exactly I should reach out to the policy support team in this screen, but I guessed it was via the arrow to the right of the "Appeals" section. I specifically mentioned that this permission had been removed, but the "policy declaration" could not be removed - there is no provision on the console to do that.

Again I got a template email as a reply saying "we are unable to approve your app's use of REQUEST_INSTALL_PACKAGES", but at the bottom of that email, there was some additional information -

Step 2: Submit a compliant update or remove the permission from your app

Once you’re ready to submit a compliant version of your app:

1. Make the necessary updates to address the issue(s) identified above OR please remove the permission from your manifest AND in-app functionality.
2. Double check that your app is compliant with all other Developer Program Policies.
3. Sign in to your Play Console, upload the modified, policy compliant APK across all tracks, and deactivate the non-compliant APK(s). 
   1. To deactivate a non-compliant APK, please create a new release and upload a compliant APK to the same track.
   2. Be sure to increment the APK version number and set the release to 100% rollout, in order to successfully override and deactivate the non-compliant APK. 
4. To save any changes you make to your release, select Save.
5. When you've finished preparing your release, select Review release.

Kindly note that your changes aren't sent for review automatically. You must go to the Publishing overview page and click Send for review to submit your changes.

Aha - so I have to deactivate non-compliant APKs on all tracks - not just the Production track. And how should I deactivate?

https://stackoverflow.com/questions/48624580/how-to-deactivate-my-previous-version-retained-apk-in-play-store

So I have to go to each track and add a new version with "Create New Release". For that, I had to create one more version of the "fixed" APK, since the current "proper" release had been auto-rejected - I had to build another apk with an incremented version code. One of the rejected apks had a higher version code than the current "fixed" version. So, I added one to that to get my next version code - though retaining the same version name and version number displayed inside the app, which have to be changed via config.xml and moodle.config.json. 

For changing only the version code, I just did Edit --> Find --> "Replace in files" inside Android Studio for the old version code with the new version code, with "Scope" as the "search in" parameter. It found 7 matches in 6 files or something like that. 7 clicks of "Replace" later, clicking on "Generate signed APK" got me the fresh apk which I could upload to the Play store. Much easier than going through the entire Cordova build process. 

Now, after submitting all these tracks, it looks like the update has not been auto-rejected. I also edited the "Policy declaration" mentioning that this permission is no longer needed. Perhaps after the update is reviewed, the policy declaration change - where this permission is not listed - would also happen. We have to wait and see. Maybe a week.

Edit: The update went through in 2 days.

failed attempts to build the moodle app apk

 Documenting my failed attempts to build the Moodle Android app.

Edit: The successful attempt is documented in this post.

For customizing
We may need to follow
http://blog.vinodsingh.com/2020/05/how-to-customize-moodle-mobile-app.html

But even the basic moodleapp fails to build - as below.
Android Studio version etc as of March 4th to 7th 2021.


Steps for build
---------------

Following https://docs.moodle.org/dev/Setting_up_your_development_environment_for_Moodle_Mobile_2
except for some exceptions, and ignoring errors.

wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash




nvm install node
# don't do nvm use 11 - current version is 15, that works. 11 has bugs.

sudo apt-get install libsecret-1-dev

git clone https://github.com/moodlehq/moodleapp.git moodleapp
cd moodleapp
git checkout integration

# dont't run npm run setup - run each individually, since errors can be ignored.
npm install 
#(
#npm ERR! npm ERR! network aborted
#npm ERR! npm ERR! network This is a problem related to network connectivity.
#Re-ran ...
#)
#Took ~ 10 minutes on lenovoPC, 2 minutes on GCP.

npx cordova prepare 
# ignore the errors.
#Failed to restore plugin "phonegap-plugin-push". You might need to try adding it again. 
# Error: CordovaError: Failed #to fetch plugin 
# git+https://github.com/moodlemobile/phonegap-plugin-push.git#moodle-v3 via registry.
#Probably this is either a connection problem, or plugin spec is incorrect.
#Check your connection and plugin name/version/URL.
# (Ran again, due to internet breakage)


npx gulp

npm start
# Did not show any errors, showed waiting for connection

npx ionic cordova platform remove android
npx ionic cordova platform remove ios
npx ionic cordova platform add android
# did not do npx ionic cordova platform add ios
# ignored the error, 
# Failed to fetch platform cordova-android@^9.0.0
#Probably this is either a connection problem, or platform spec is incorrect.
# in verbose mode, 
#npm ERR! errno ENOTFOUND
#npm ERR! network request to https://registry.npmjs.org/coffeescript failed, 
#reason: getaddrinfo ENOTFOUND #registry.npmjs.org
#npm ERR! network This is a problem related to network connectivity.
# Ran again, 
# npx ionic cordova platform add android --verbose
# Platform android already exists.
# did remove and add again, then looks like it is not a connection problem, but platform spec is incorrect

# https://stackoverflow.com/questions/55965450/failed-to-fetch-platform-cordova-android8-0-0
# ignoring,

npm run dev:android
# gave error, but wrote
#cordova-android-support-gradle-release: Android platform: V7+
#cordova-android-support-gradle-release: Wrote custom version '27.1.0' to 
#/home/user/moodleapp/platforms/android/app/#build.gradle
#cordova-android-support-gradle-release: Wrote custom version '27.1.0' to 
#/home/user/moodleapp/platforms/android/
#cordova-android-support-gradle-release/moodlemobile-cordova-android-support-gradle-release.gradle
#[ERROR] An error occurred while running subprocess cordova.

# on GCP, error was No valid Android SDK root found.
# did again on a fresh terminal so that .bashrc is processed again,
# then it worked.

sudo apt-get install gradle
sudo apt-get install libgradle-android-plugin-java

Edits as per
https://docs.moodle.org/dev/Setting_up_your_development_environment_for_Moodle_Mobile_2#Compiling_using_AOT

npm run ionic:build -- --prod
# LenovoPC bogged down since it uses too much memory and goes into swap, trying on cloud dev machine.
# GCP finished in 6 minutes - used nearly all of 16 GB RAM. 200% cpu of 4 core machine.
npx cordova run android

# on GCP, gave response as below
# npx cordova run android
Conflict found, edit-config changes from config.xml will overwrite plugin.xml changes
9.0.0
cordova-android-support-gradle-release: Android platform: V7+
cordova-android-support-gradle-release: Wrote custom version '27.1.0' to 
/home/user/moodleapp/platforms/android/app/build.gradle
#cordova-android-support-gradle-release: 
#Wrote custom version '27.1.0' to 
#/home/user/moodleapp/platforms/android/cordova-android-support-gradle-release/moodlemobile-cordova-android-support-gradle-release.gradle
Cannot find module 'xcode'
Require stack:
- /home/user/moodleapp/plugins/cordova-plugin-add-swift-support/src/add-swift-support.js
- /home/user/moodleapp/node_modules/cordova-lib/src/hooks/HooksRunner.js
- /home/user/moodleapp/node_modules/cordova-lib/src/plugman/install.js
- /home/user/moodleapp/node_modules/cordova-lib/src/plugman/plugman.js
- /home/user/moodleapp/node_modules/cordova-lib/cordova-lib.js
- /home/user/moodleapp/node_modules/cordova/src/help.js
- /home/user/moodleapp/node_modules/cordova/src/cli.js
- /home/user/moodleapp/node_modules/cordova/bin/cordova

###########

# so tried importing into android studio
# https://cordova.apache.org/docs/en/10.x/guide/platforms/android/index.html#debugging
# 

#android studio gave error as follows:
# ~/androidstudioerr.txt
#Manifest merger failed : uses-sdk:minSdkVersion 19 cannot be smaller than version 22 declared in library [:CordovaLib] 
#/home/user/moodleapp/platforms/android/CordovaLib/build/intermediates/library_manifest/debug/AndroidManifest.xml 
#as the library might be using APIs not available in 19
#	Suggestion: use a compatible library with a minSdk of at most 19,
#		or increase this project's minSdk version to at least 22,
#		or use tools:overrideLibrary="org.apache.cordova" to force usage (may lead to runtime failures
#
# https://github.com/apache/cordova-android/issues/1070
# so edited config.xml with the info as given in the above, but that gave unbound prefix error.
# 
# Then, tried changing minSdkVersion=22 (from 19) on line 212 of config.xml
# still
# so, https://www.learningsomethingnew.com/how-and-why-to-change-your-android-cordova-apps-min-sdk-version
# but cordova platform rm android
# cordova: command not found.
# so tried
npx ionic cordova platform remove android
npx ionic cordova platform add android
# but the same error inside android studio.

Edit: The successful attempt is documented in this post.

customizing and building the moodle app

After learning how to use an older node version, the moodle app could finally be customized and built - a follow-up to my failed trials here. 

The customization steps and the relevant modified files are in this location, with the changes being mentioned in the readme as follows:

1. Changes as per the moodle doc on compiling with AOT
Edit Oct 2022 - the link content seems to have changed, so here is the archived version.

2. Change to the build.gradle to force minSdkversion 22

ext.cdvMinSdkVersion = 22

to build with android studio. This can also be done in config.xml

3. Changing all ~ and ^ package dependencies in package.json (except cordova-android itself) to the exact package. Done by replacing

"~ with "

and

"^ with "

in package.json

Or else, all sorts of dependency issues while building the 2019 release in 2021. (The nvm use 11 may also suffice for solving the dependency issues.)

The build directory takes up more than 1 GB, lots of dependencies are downloaded. 

4. Customization - Changes as per the Configuration heading at this post,

config.xml

reduced SplashScreenDelay

and SplashShowOnlyFirstTime true

also, very important,

content src="https://sssvidyavahini.org" would make the app like a simple webview.

We don't want that, so leave the content src as it is, 

but change the siteurl in src/config.json

and onlyallowlistedsites true.

We also set android-minSdkVersion" value="22" in config.xml

google-services.json - changed the package name

and changed the version in src/config.json , ensured it is different, but same number of digits.

5. Changing logo and splash screen as per above url and also resources/android/icon-background.png and icon-foreground.png

and in src dir assets/img

Build steps - 

Set up environment variables as per

https://cordova.apache.org/docs/en/10.x/guide/platforms/android/

JDK tar.gz needed oracle login, created as my official email

Set JAVA_HOME

https://docs.oracle.com/cd/E19182-01/821-0917/inst_jdk_javahome_t/index.html

in .bashrc, added the following.

export JAVA_HOME=/home/mac/Downloads/jdk1.8.0_281
export PATH=$JAVA_HOME/bin:$PATH
export ANDROID_SDK_ROOT=/home/mac/Android/Sdk
export PATH=$ANDROID_SDK_ROOT/tools:$ANDROID_SDK_ROOT/tools/bin:$ANDROID_SDK_ROOT/platform-tools:$PATH

As mentioned in the cordova documentation, we can open a cordova project inside android studio for the final build - choose the import gradle project option, and choose the platforms/android directory.

But we need to edit www folder outside android studio, then copy over changes by doing cordova build - in our case, npx etc as below.

The build process is detailed in the following steps to build document:

# https://docs.moodle.org/dev/Setting_up_your_development_environment_for_Moodle_Mobile_2

nvm install node 11.15.0

# https://www.sitepoint.com/quick-tip-multiple-versions-node-nvm/

nvm use 11

# this is very important - if starting again in another terminal window, must do nvm use 11

# or else all sorts of hard to diagnose errors occur during the build.

sudo apt-get install libsecret-1-dev

npm install 

# cordova.plugins.diagnostic: Diagnostic plugin - ERROR: ENOENT: no such file or directory, open '/home/mac/StudioProjects/LMSappBuildTrial/config.xml'

# added 1829 packages from 1022 contributors and audited 1958 packages in 98.592s

npx cordova prepare 

# Current working directory is not a Cordova-based project.

# https://stackoverflow.com/questions/21276294/cordova-current-working-directory-is-not-a-cordova-based-project

#mkdir www

npm install cordova

# this is also important. Without this, the www directory is not populated for the final prod build.

npm i -g cordova-res

# This is also important - without this, the image resources won't get processed

# config.xml was missing, so copied over with git

# No platforms added to this project. Please use `cordova platform add <platform>`.

npx ionic cordova platform add android --verbose

#Source path does not exist: resources/android/icon/drawable-hdpi-smallicon.png

#Error: Source path does not exist: resources/android/icon/drawable-hdpi-smallicon.png

# editing gitignore file to add the resources

npx cordova prepare

# ignoring the warning about conflict

npx gulp

npm start

# This is an optional step to check in a browser and verify.

# waiting till transpile started etc. then Ctrl+C

# If we wait till the end - nearly an hour on a 4 GB system? it opens in the default browser.

# https://docs.moodle.org/dev/Setting_up_your_development_environment_for_Moodle_Mobile_2#Compiling_using_AOT

cp -v "changes made to moodleapp files/inside node_modules dir/@angular/platform-browser-dynamic/esm5/platform-browser-dynamic.js" "node_modules/@angular/platform-browser-dynamic/esm5"

cp -v "changes made to moodleapp files/inside node_modules dir/@ionic/app-scripts/dist/util/config.js" "node_modules/@ionic/app-scripts/dist/util/config.js"

# edited gitignore to not ignore this config.js, did git add -f.

npm run ionic:build -- --prod

# this takes a lot of RAM and a lot of time. 

# if possible, avoid swapping by closing all apps and clearning memory before doing this.

# REMEMBER to use nvm use 11 if doing this in a fresh terminal

#PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND 

# 3682 mac       20   0 3142252 2.215g  28948 R 131.5 59.6   6:34.48 node

#   build prod finished in 3128.04 s - this was on a 4GB machine

# on a 16 GB GCP instance, it took up around 1/3rd of memory - 5 GB+ - 

# and finished in just 5 or 6 minutes instead of 50 minutes.

# npx cordova run android

# Could not find an installed version of Gradle either in Android Studio,

#or on your system to install the gradle wrapper. Please include gradle 

#in your path, or install Android Studio

# So, I just imported the platforms/android directory using the Import Gradle project option in Android Studio 4.1.3, and built from there.

changes after upgrading to Moodle 4

Copy-pasting edited extracts from an email exchange after upgrading a test server to Moodle 4 - 

... the orange band appears with company logo. Need to check how to remove orange band or replace the logo with SSSVV.

 

Also the footer information is not displayed like in the present case.

The easy way to remove it is to add the following to the Advanced tab in Moove settings,  Raw SCSS - 

footer#page-footer .copyright  {display:none;}

which I have done now. 

 

a) In the footer when you click on the mailbox it goes to MS edge account and not to the given mail id.

The footer link for contact us is a mailto: link. This will work properly if your browser is configured properly to handle mailto: links.

If you find that clicking that link takes you to your microsoft account, you might be using microsoft edge browser? In that case,

https://christianheilmann.com/2020/07/20/quick-tip-how-to-make-mailto-links-open-in-gmail-in-microsoft-edge/

 

b) Phone when clicked goes to a number which has this number added before the number given "662453:"

Just like the contact us email link is a mailto: link, the telephone link is a tel: link, which again needs to be configured correctly in your browser for it to work.

The incorrect behaviour is also because the tel: link is not configured properly. Only the number should be entered, and not text like "Mobile:".

https://www.elegantthemes.com/blog/wordpress/call-link-html-phone-number

 

c) Although I have a Web whatsapp installed on my laptop it says the link is wrong. On mobile it works.

One workaround would be to enter the whatsapp phone number as

00919876543210 - without spaces and without the + symbol, using 0091 instead of +91 - then I think it works.

 

d) When one clicks on Google Play icon for mobile app. the user cannot return to the front page but goes to the site home page. To return to the front page he has to login and logout.

I think this is related to what we have discussed earlier, Moodle shows a different page when one is logged in as compared to when logged out. Also, user is automatically logged in to guest login if any of the internal links which have guest access are clicked.

 

e) Is it possible to remove the App. Store link or the version 4 of moodle supports mobile app. on IOS devices.

Yes, we can remove if necessary using custom CSS. We probably need to link to the India site, like

https://apps.apple.com/in/app/moodle/id633359593

(the US site https://apps.apple.com/us/app/moodle/id633359593 does not seem to load for users in India)

7) I have added sample videos in some blocks to check. However if you open a video for viewing one cannot return to the front page but lands on the site home page. (Front page means non logged in page). One has to log in and log out to land again on the front page.

Please see how the marketing block looks like on the production site. I don't think there is any change in functionality 

https://oursite.org/admin/settings.php?section=themesettingmoove#theme_moove_frontpage

 Whenever I log into the site I get a mail saying new sign in. May be a version 4.0 addition. Can it be disabled or it will remain as default.

You can probably change this behaviour from

https://oursite.org/message/notificationpreferences.php

 

 

Building the cordova-based Moodle app v4.0.2 on Github Actions

It turns out that building the latest version (4.0.2) of the Moodle app on github actions is much easier than on travis-ci, because

• The Travis build environment seems to have Java 8 by default, and the latest version of the Android SDK (for SDK level 31 and above) need Java 11 
• Github actions seem to have all the latest build tools etc pre-installed.

The github actions yml file, which has all the commands necessary for a debug build, is at 
https://github.com/hn-88/LMSapp/blob/main/.github/workflows/buildSSSVV.yml 

Can refer to Notes on building the cordova-based Moodle app for Android for more info on building release version, etc.

Edit: In my implementation, I used manually triggered actions - https://levelup.gitconnected.com/how-to-manually-trigger-a-github-actions-workflow-4712542f1960 - by having the line
on: workflow_dispatch

moodle app build for v4.3.0

Some small changes were needed in the github workflow in order to build the latest version, 4.3.0, of the moodle android app. My earlier posts about building the moodle app can be found here. 

Trying the old workflow led to some errors like

Failed to restore plugin "cordova-plugin-screen-orientation". You might need to try adding it again. Error: Error: Command failed with EACCES

Checking the testing.yml in the 4.3.0 tag found some extra lines like 

npm ci --no-audit --prefix cordova-plugin-moodleapp

Adding that led to a different error,

 cordova-plugin-moodleapp@0.0.0 prod /home/runner/work/LMSapp/LMSapp/cordova-plugin-moodleapp

> NODE_ENV=production scripts/build.js

sh: 1: scripts/build.js: Permission denied

Trying the simplistic solution of giving run permissions for everything,

chmod -R +x ./*

This worked, except for needing resources/android/icon-background.png to be added manually as in earlier releases. 

Tweaked the build script to directly pull the codebase of the release from github instead of manually adding it, with all customizations from our private repo which has the following file structure:

 

 

 

app with notification support - details about Moodle notifications

Copy-pasting from some correspondence about a Moodle site and notifications desired from it, by email and by mobile app notifications - 

1)     Site announcements - The admin user has to go to https://site.org/admin/user/user_bulk.php and choose "all users" and then choose "Send a message" from the dropdown at the bottom.

2)     Calendar events on site - This requires a plugin - https://docs.moodle.org/311/en/Event_reminders

https://docs.moodle.org/39/en/Moodle_app_notifications#Event_Reminders_plugin

3)     Feedback to designated teacher - Yes, if the teacher enables notifications for this on mobile preferences as well as on the website at https://site.org/message/notificationpreferences.php

4)     Submission of assignments to training admin or designated teacher - Yes, if the teacher enables notifications for this on mobile preferences as well as on the website at https://site.org/message/notificationpreferences.php

5)     Mobile messaging one to one and in group - Yes, if the user enables notifications for this on mobile preferences as well as on the website at https://site.org/message/notificationpreferences.php

6)     Completion of activity to training admin/designated teacher - There is a notification for Quiz completion. Not all activities have it - Yes, if the teacher enables notifications for this on mobile preferences as well as on the website at https://site.org/message/notificationpreferences.php

7)     Course level announcements - The teacher has to send a message to all participants using the method at https://docs.moodle.org/311/en/Messaging#Sending_a_message_to_selected_course_participants . If the user has enabled notifications, they will get notifications if the teacher sends them via this method.

8)     Course level calendar events - This requires a plugin - 
https://moodle.org/plugins/local_reminders

9)     Scheduled chat sessions if installed - The teacher has to send a message to all participants using the method at https://docs.moodle.org/311/en/Messaging#Sending_a_message_to_selected_course_participants .

10)  One to one messaging within a course - Yes, if the user enables it, and if it is enabled by the admin site-wide.

11)  Group messaging within a course - Yes, if the username is known, if not disabled by administrator,

https://docs.moodle.org/311/en/Messaging

12)  Badge/certificate issued - Yes, if the user enables notifications for this on mobile preferences as well as on the website at https://site.org/message/notificationpreferences.php

long back-and-forth about Moodle quiz content not working

After an upgrade to Moodle 4.5.3, the admin of one of our Moodle instances complained about some content not working. It finally turned out to be due to the use of deprecated mod_questionnaire and probably some caching. Copy-pasting large parts of our conversation.

We are experiencing delays in server from yesterday and today.

It is unresponsive when we try to add contents to our upcoming module in <course> which may start tomorrow.

Logged in to the site OK. As I have mentioned before, please add content in small enough chunks to avoid timeouts.

The site was up after some time. But with no heavy activity still we find it not responsive. Particularly that involves heavy database calls like the quizzes. This happened yesterday and even today.

e.g., https://ourdomain.org/mod/quiz/view.php?id=5032 which has the quiz section, is not loading at all. 

Can you check if the database server is  healthy for <course>?

1. The link which you have sent does not load due to an invalid course module id - "Invalid course module ID: 5013 ... More information about this error"

2. Copy-pasting from an email I had sent to the team via SG on 11 Jan 2023 - 

Some suggestions to make the course(s) load faster:

1. Currently, if you click on one of the courses, a huge page, with many images, for all weeks, loads in the browser. We can make it load a bit faster using Cloudflare, but such a large page is bound to cause problems with older computers / machines with less RAM / on slower networks / etc. It would be better to show a collapsed view, with each week opening up only on clicking. (the collapse all link on the top right). Or on separate pages.

2. As I mentioned, please use jpg quality = 70 to 75 instead of png. 

3. I'll make the cloudflare changes to turn proxying on. 

4. Clicking on question bank also loads up 1000 questions at once. This would cause problems as the number of questions / size of each question increase. It would be desirable to load up fewer questions - maybe 100 at a time as with the link at the bottom, "Show 100 at a time"

<the link> This was working on Friday night. This is a valid link for the quiz section of module 8.

Discovered that quiz sections are not opening for previous modules also - which the participants successfully faced for the last 7 modules. All quizzes and question bank resources were working fine till last Friday. So there is something suspicious in the database server. 

We need all the previous module results and also the new quizzes that are to be opened tomorrow.

Can you please check the health of the app and database server?

Moodle is installed and the database is running, which is how we are able to log on, see course pages, etc. - so the "health of the app and database server" are fine. 

Digging deeper, in the plugins page, https://ourdomain.org/admin/plugins.php

I have seen some plugins "missing from disk" errors. These would have been "missing" since the Moodle upgrade of Mar 18.

Just to be safe, I have now copied these "missing from disk" plugins to the moodle plugins directory. It may be that these plugins are no longer supported, hence Moodle removed them - other plugins prompt for upgrades, which have been applied.

I have also turned off cloudflare caching, so that you will get more time before server timeout. But this may make pages slower to load for those taking the courses, so once you are done with admin tasks requiring longer db queries, please let me know, and I will again turn on cloudflare caching for faster response for the users of the course.

The link which you sent still gives the same error.

I'm not sure what else I can help you with. If you want raw database backups from the weekly backups, I can send you.

Do you have the list for missing plugins and updated ones? That can give some ideas. 

local/welcome
local/mailtest
report/customsql
blocks/accessibility
lib/editor/atto
mod/questionnaire 

Or can we revert to the last working version of Moodle? The troubleshoot time is less and we need to get the module ready for tomorrow morning at 4am.

Note: Also in general, the site is too slow when compared to a couple of weeks before.

As far as I know, Moodle is working fine. So, as far as I'm concerned, "last working version of Moodle" is the current live site.

I can put the site back to last week's or the week before's - but then, any responses from the people taking from the course or any other changes made on the site after last week or 2 weeks back would be wiped out. Do you really want to do that? I don't think that's a good idea.

Yes, the site would be slower now that I have removed cloudflare caching as mentioned in the previous email.

Thanks for the list. `mod/questionnaire` seems to be related. But we can wait and see if we can create another set of quiz sections.

On software upgrade -> yes if things dont work, we need to revert to the last working version. Not yet decided. Shall keep you updated as we are trying to create a new section.

(and 20 minutes later,)

Surprisingly things are working again.. Not sure how but it looks fine now. Also the site seems to be a bit faster than before. Not sure how?

No changes were done from our side.

Moodle does some scheduled tasks - which are listed under Site Administration > Server > Tasks > Scheduled tasks

My guess would be that after I restored the missing plugin files, some automated task ran at some point of time to either clear database cached replies, or to fix broken links or something like that. 

Anyway, all's well that ends well. But please don't put too much content on a single page - that would cause the site to load slowly. Please see Course Format > Course Layout > Show all sections on one page and change it to "Show one section per page" on each of the courses you're working on. That will make pages load much faster.

5 days later, at 9 pm,

again the site is too slow from this afternoon 12PM. Even a single module  in the course page is taking minutes to load. Can you please check if there are any background jobs running?

This is affecting our schedule as we wanted to open a new module tomorrow and we are stuck for long time today and unable to make it up.

After a long time I am getting this (Error writing to database popup) - when trying to create a quiz (just a duplicate of an existing one)

Yes, Moodle runs lots of background jobs all the time. You can check these by going to 

https://ourdomain.org/admin/tasklogs.php

Right now, the server loads are very low, but I did get a db server load high alert (85%, as against a healthy load of up to 80%) at 10:15 am yesterday.

 Additionally, if many of your users are logged in, that can also slow down your response times. One possible mitigation in such cases would be for you to lock out other users by going to Maintenance Mode

https://ourdomain.org/admin/settings.php?section=maintenancemode

choose "Enable" and save - this will prevent other users from using the server - do your maintenance tasks, then go to the same page, and "Disable" - after which other users can continue to work.

And as I have mentioned earlier, running large db (database) queries will slow down the server and or make it time out. If that happens, please break down the action you wish to take into smaller chunks. "Just making a duplicate of a quiz" can be very db intensive if the quiz has a lot of content, multimedia content, a lot of users, a lot of responses, multimedia responses, and so on.

Attempted to create a blank quiz and it fails like below. (Screen showing "recalculating grades" and error writing to database at the end.) This does not have any questions. So seriously something is wrong with the backend.

Doing a search,

https://www.bing.com/search?pglt=641&q=creating+a+quiz+recalculating+grades+timeout+mod+questionnaire+moodle

Found

https://moodle.org/mod/forum/discuss.php?d=401173

https://tracker.moodle.org/browse/MDL-49324

So, this is possibly a bug. But I cannot fix Moodle bugs for you.

Not sure how I can help you with this. I can turn off cloudflare caching so that you will get a bit longer before timeout. (Found that this is already done.)

 Then you can try putting the site in maintenance mode and then try creating the quiz. But if that also fails, I have no further ideas.

Thanks for the info bro. The site was noticeably faster a couple of weeks ago. 

Managed to create the sections for this module. Hopefully things get faster to load for users tomorrow.