Saturday, 29 July 2017

Rolling kernels, target processors and Peppermint OS in latest 'isorespin.sh' update


I've updated my 'isorespin.sh' script to include the following concepts:

  • rolling kernels
  • target processors

and I've additionally included support for Peppermint OS (an Ubuntu kernel based distro).

The  need for rolling kernels  is primarily to address the shortfall that existing ISO kernels typically don't have the hardware support required for the latest devices. They ship with the 'kernel of the day' whereas the latest hardware tends to require, well, the latest kernel to fully work. Ubuntu have in part addressed this through their LTS Enablement Stacks. Another alternative to get 'newer' kernels is to enable the proposed repository however there is also a pre-release and test kernel repository together with the unstable repository where mainline kernels are migrated into Ubuntu as well as the upstream mainline kernels.


I've added an option '--rolling-list' to keeping track of what new kernels are available where with options to easily incorporate them when respinning an ISO. I've termed the kernel types as release, proposed, testing and unstable to reflect the repositories they are drawn from as described above. And if it is not abundantly obvious but any kernel other than the formally released ones are not encouraged for anyone needing a stable system or anyone who is not comfortable running into occasional or even frequent breakage.

The other concept I've introduced is respinning an ISO for a target processor. Whilst I've documented how to respin an ISO and what to include it does require reading. So now by including the option of '--atom' or '--apollo' you'll get an ISO that hopefully works on devices with the respective Intel SoCs. The options simply include the flags, packages, scripts and commands that I recommend and mirror the manual invocations of:

For '--atom': -l rtl8723bX_4.12.0_amd64.deb -f linuxium-install-UCM-files.sh -f wrapper-linuxium-install-UCM-files.sh -f linuxium-install-broadcom-drivers.sh -f wrapper-linuxium-install-broadcom-drivers.sh -c wrapper-linuxium-install-UCM-files.sh -c wrapper-linuxium-install-broadcom-drivers.sh

For '--apollo': -b rEFInd

The files are automatically downloaded from the internet if they are not found as part of the command invocation and this should simplify respinning for those looking something quick and easy.

Both these new functionality require that you have the 'curl' package installed prior to running the respin script however the script will check and warn you if it is missing.

The new syntax for 'isorespin.sh' is:

Usage: /usr/local/bin/isorespin.sh [ -h | -v | --rolling-list ]
       /usr/local/bin/isorespin.sh -i <ISO> [ [ -u | -k <kernel> ] | -r "<repo> ... " | -p "<pkg> ... " | -l "<pkg.deb> ... " | -f "<file> | <directory> ... " | [ -s <size>MB | GB ] | [ -b GRUB | rEFInd ] | ...
       /usr/local/bin/isorespin.sh ... -w <directory> | -d "<pkg> ... " | -e "<pkg> ... " | -c "<cmd> ... " | -o "<file> | <directory> ... " | -g "" | "<kernel boot parameter> ... " | ...
       /usr/local/bin/isorespin.sh ... --apollo | --atom | ...
       /usr/local/bin/isorespin.sh ... --rolling-release | --rolling-release-hwe | --rolling-release-hwe-edge | --rolling-proposed | --rolling-proposed-hwe | --rolling-proposed-hwe-edge | ...
       /usr/local/bin/isorespin.sh ... --rolling-testing | --rolling-testing-hwe | --rolling-testing-hwe-edge | --rolling-unstable | --rolling-unstable-hwe | --rolling-unstable-hwe-edge ]

and the options have been included in the GUI:





I've also added Peppermint OS support. This OS uses an Ubuntu kernel and is particularly suitabale for low specification devices like the original Ubuntu Intel Compute Stick which only has 1GB RAM and 8GB storage.


I was able to watch a YouTube video at 1080p via wifi which is pretty impressive for that device.



Mainline kernel v4.13 is progressing with RC2 released this week.


As the opt-in Ubuntu flavours released their Artful 17.10 Alpha 2 ISOs yesterday I've respun the Lubuntu one with "--atom -u" which generated the log:

Script '/usr/local/bin/isorespin.sh' called with '-i lubuntu-17.10-alpha2-desktop-amd64.iso --atom -u' ...
Work directory 'isorespin' used ...
ISO '/home/linuxium/lubuntu-17.10-alpha2-desktop-amd64.iso' respun ...
Bootloader 'GRUB' added ...
Kernel updated with mainline kernel version '4.13.0-041300rc2-generic' ...
Local package '/home/linuxium/isorespin/rtl8723bt_4.12.0_amd64.deb' added ...
File '/home/linuxium/isorespin/linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/linuxium-install-broadcom-drivers.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-broadcom-drivers.sh' added ...
Command run ...
# wrapper-linuxium-install-UCM-files.sh
./linuxium-install-UCM-files.sh: Extracting UCM files ...
./linuxium-install-UCM-files.sh: Installing UCM files ...
./linuxium-install-UCM-files.sh: Reloading UCM driver ...
./linuxium-install-UCM-files.sh: Installation of UCM finished 
# wrapper-linuxium-install-broadcom-drivers.sh
./linuxium-install-broadcom-drivers.sh: Extracting Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Reloading Broadcom driver ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom bluetooth service ...
Created symlink /etc/systemd/system/multi-user.target.wants/brcmbt.service -> /lib/systemd/system/brcmbt.service.
./linuxium-install-broadcom-drivers.sh: Starting Broadcom bluetooth service ...
Running in chroot, ignoring request.
./linuxium-install-broadcom-drivers.sh: Installation of Broadcom finished 
Respun ISO created as 'linuxium-v4.13-rc2-lubuntu-17.10-alpha2-desktop-amd64.iso'.

and the respun ISO can be downloaded from here.


I've also respun the Ubuntu 17.04 ISO and added a 200MB persistent partition meaning the ISO which now supports Intel Atom devices with also boot on Intel Apollo devices:

Script '/usr/local/bin/isorespin.sh' called with '-i ubuntu-17.04-desktop-amd64.iso -u --atom -s 200MB' ...
Work directory 'isorespin' used ...
ISO '/home/linuxium/ubuntu-17.04-desktop-amd64.iso' respun ...
Kernel boot parameters 'persistent' added ...
Bootmanager 'rEFInd' added ...
Kernel updated with mainline kernel version '4.13.0-041300rc2-generic' ...
Local package '/home/linuxium/isorespin/rtl8723bs_4.12.0_amd64.deb' added ...
File '/home/linuxium/isorespin/linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/linuxium-install-broadcom-drivers.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-broadcom-drivers.sh' added ...
Command run ...
# wrapper-linuxium-install-UCM-files.sh
./linuxium-install-UCM-files.sh: Extracting UCM files ...
./linuxium-install-UCM-files.sh: Installing UCM files ...
./linuxium-install-UCM-files.sh: Reloading UCM driver ...
./linuxium-install-UCM-files.sh: Installation of UCM finished 
# wrapper-linuxium-install-broadcom-drivers.sh
./linuxium-install-broadcom-drivers.sh: Extracting Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Reloading Broadcom driver ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom bluetooth service ...
Created symlink /etc/systemd/system/multi-user.target.wants/brcmbt.service -> /lib/systemd/system/brcmbt.service.
./linuxium-install-broadcom-drivers.sh: Starting Broadcom bluetooth service ...
Running in chroot, ignoring request.
./linuxium-install-broadcom-drivers.sh: Installation of Broadcom finished 
Persistence partition of '200MB' added ...
Respun ISO created as 'linuxium-persistence-v4.13-rc2-ubuntu-17.04-desktop-amd64.iso'.

and the respun ISO can be downloaded from here.

Anyone interested in running an ISO shown in the opening image can just respin the latest daily Ubuntu release with '--rolling-unstable --atom' options. Simple!

Please donate if you find the script useful using the following link http://goo.gl/nXWSGf as everything helps with development costs.

34 comments:




  1. Here is the script to build into the iso Intel 01.org 3d Accell Support

    #!/bin/bash
    cd /tmp
    #This is for ubuntu 17.04
    #Install Intel Xorg Hardware accelerate Drivers and setup
    wget https://download.01.org/gfx/RPM-GPG-GROUP-KEY-ilg
    echo 'Section "Device"' >> /etc/X11/xorg.conf
    echo ' Identifier "Intel Graphics"; >> /etc/X11/xorg.conf
    echo ' Driver "intel"' >> /etc/X11/xorg.conf
    echo 'EndSection' >> /etc/xorg.conf
    apt-key add RPM-GPG-GROUP-KEY-ilg
    echo "deb https://download.01.org/gfx/ubuntu/17.04/main zesty main" >> /etc/apt/sources.list
    apt-get -y update
    apt-get -y upgrade i965-va-driver libva-egl1 libva-glx1 libva-tpi1 libva-wayland1 libva-x11-1 libva1 vainfo intel-gpu-tools libunwind8
    apt-get -f --fix-broken install
    apt-get --reinstall install vainfo intel-gpu-tools libunwind8 xserver-xorg-video-intel
    #Finnished Installing Intel Drivers for 17.04

    #Update and clean
    apt-get -y upgrade
    apt-get -y clean
    rm /var/cache/apt/packages/lock

    #Install Chrome and enable full Intel Hardware accelerated
    mv /usr/bin/google-chrome /usr/bin/chrome
    echo "google-chrome --ignore-gpu-blacklist --enable-vaapi --enable-gpu-rasterization --enable-native-gpu-memory-buffers" > /usr/bin/google-chrome
    chmod 755 /usr/bin/google-chrome

    ReplyDelete
    Replies
    1. Remember to do the following since the script enables 3d accell in chrome too

      wget https://download.01.org/gfx/ubuntu/17.04/main/pool/main/i/intel-graphics-update-tool/intel-graphics-update-tool_2.0.5_amd64.deb

      wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb

      and add the line
      -c wrapper-intel3dchrome.sh
      -l google-chrome-stable_current_amd64.deb
      -l intel-graphics-update-tool_2.0.5_amd64.deb

      P.S. If you need a working goodix driver for the latest rc-2 reply and ill send you it.

      Delete
  2. Here is what chrome looks like by default.
    https://preview.ibb.co/hiotN5/3dnoaccel.jpg

    Here is what chrome looks like full HW 3d Acceleration.
    https://preview.ibb.co/eSKiN5/3daccel.jpg

    ReplyDelete
    Replies
    1. Used with the latest mainline kernel and I'm not seeing any benefit with the Intel driver vs without. And this is on a first gen Intel Commpute stick running 1080p YouTube videos. Maybe different behaviour on other devices or is the drm-intel-nightly a mandatory prerequisite?

      Delete
    2. I was talking about the 01.org intel video drivers. Not the drm-intel-nightly. On my device I could not play fullscreen video without the intel video drivers.

      Delete
    3. Interesting. As shown by the video screenshot above I've not encountered the same issue although that example was using Firefox.

      I modified your script (saved as 'wrapper-intel-ha-chrome.sh') with minor corrections:

      #!/bin/bash

      # Linuxium's isorespin.sh wrapper script to install Intel's Xorg Hardware Acceleration driver specifically for Ubuntu 17.04
      # Based on Jason Kurtz's wrapper-intel3dchrome.sh script

      # Install driver configuration and packages
      wget --timeout=10 https://download.01.org/gfx/RPM-GPG-GROUP-KEY-ilg -O RPM-GPG-GROUP-KEY-ilg
      apt-key add RPM-GPG-GROUP-KEY-ilg
      rm -f RPM-GPG-GROUP-KEY-ilg
      echo -e 'Section "Device"' >> /etc/X11/xorg.conf
      echo -e '\tIdentifier\t"Intel Graphics"' >> /etc/X11/xorg.conf
      echo -e '\tDriver\t"intel"' >> /etc/X11/xorg.conf
      echo -e 'EndSection' >> /etc/X11/xorg.conf
      echo "deb https://download.01.org/gfx/ubuntu/17.04/main zesty main" >> /etc/apt/sources.list
      apt-get -y update
      apt-get -y upgrade
      apt-get -y install i965-va-driver libva-drm1 libva-egl1 libva-glx1 libva-tpi1 libva-utils libva-wayland1 libva-x11-1 libva1 vainfo

      # Clean up
      apt-get -y clean
      apt-get autoclean
      apt-get -y autoremove

      # Clone Chrome adding Hardware Acceleration
      echo "/usr/bin/google-chrome --ignore-gpu-blacklist --enable-vaapi --enable-gpu-rasterization --enable-native-gpu-memory-buffers" > /usr/local/bin/linuxium-chrome
      chmod 755 /usr/local/bin/linuxium-chrome

      # isorespin.sh -i ubuntu-17.04-desktop-amd64.iso --atom -u -f wrapper-intel-ha-chrome.sh -l google-chrome-stable_current_amd64.deb -l intel-graphics-update-tool_2.0.5_amd64.deb -c wrapper-intel-ha-chrome.sh

      so I can compare standard Chrome (as 'google-chrome') with hardware accelerated Chrome (as 'linuxium-chrome') and both play 1080p full screen videos on my Intel Z3735F device.

      The devices runs at the same temperature either with or without hardware acceleration so temperature/thermal throttling is not affecting performance. Is that the same on your device?

      Delete
    4. I am going too fully charge it and do a run time test with and without it this weekend. Ill check the temp then.

      Delete
  3. Terrific news .. Many thanks from Team Peppermint :)

    ReplyDelete
  4. When I try ubuntu,my external mouse won't work.What should I do?

    ReplyDelete
    Replies
    1. Either try another mouse or explain in more detail what the problem is including details of the type/manufacture/method of connecting etc.

      Delete
  5. Replies
    1. That is a known message when booting from a 32-bit bootloader device as then a blank screen or the normal Ubuntu splash screen appears before the device fully boots.

      Delete
  6. Bonjour.
    Pour installer le dernier noyau linux,
    créer un dossier et y copier les paquets linux-headers-****, linux-headers-***-generic et linux-image-***-generic
    Puis pour installer:
    sudo apt install ./linux-*
    Sinon, kernel panic !

    Hello.
    To install the latest linux kernel,
    Create a folder and copy packages linux-headers - ****, linux-headers - *** - generic and linux-image - *** - generic
    Then to install:
    Sudo apt install ./linux-*
    Otherwise, kernel panic!

    ReplyDelete
    Replies
    1. What device are you getting a kernel panic using v4.13-rc2 with and is it the same for v4.13-rc3?

      Delete
    2. Tablet 2 in 1 Thomson THBK1-10.64DB
      http://www.thomsoncomputing.eu/dualboot.html#

      Best Regards

      Delete
    3. Cube iwork1X. yes kernel panic with any kernel with graphic install by clicking on the package on kubuntu 17.04

      Delete
    4. By clicking on which package? Can you explain in more detail?

      Delete
    5. Après avoir installé les paquets, kernel panic. Je reboot avec un autre noyau puis installe un paquet quelconque et je reboot , plus de kernel panic donc j'ai eu l'idée d'installer les paquets rc3 en mode commande: sudo apt install ./images-*.deb ce qui indique que des commandes ne s'exécutes pas en mode installation graphique.

      After installing the packages, kernel panic. I reboot with another kernel then install any package and reboot, more kernel panic so I had the idea to install rc3 packages in command mode: sudo apt install ./images-*.deb which indicates That commands do not run in graphical installation mode.

      Delete
    6. Can you try again with the latest version of my script (v7.2.1) and see if the issue still exists?

      Delete
  7. Ian, Can you please change the isorespin script to use btrfs instead of ext4 for the persistent partition. https://btrfs.wiki.kernel.org/index.php/Main_Page

    ReplyDelete
  8. Time for a new chrome speed up I tested. By removing the media-cache and the cache. It runs alot faster on my device. Here is the chrome startup command I am using.

    google-chrome --ignore-gpu-blacklist --enable-checkered-imaging --enable-vaapi --enable-gpu-rasterization --enable-native-gpu-memory-buffers --disk-cache-dir=/dev/null --media-cache-dir=/dev/null $@

    This turns off the media and the caching for chrome. It made a big speed difference and saves disk space.

    ReplyDelete
  9. Hi again!

    Thanks for all your work. I would like to knowif this kernel should implement the audio drivers for Chuwi Lapboook 15.6inch because I have still no sound in my Laptoop. As you said 4.13+ kernel should have audio for these computers but still having problems with that.

    Thank you

    ReplyDelete
    Replies
    1. It seems the current kernel does not include the ES8316 driver by default so it needs to be manually added - see the post from Gérald TOUSSAINT on 'https://linuxiumcomau.blogspot.com.au/2017/07/sixth-look-at-ubuntu-1710-julyshakedown.html'.

      Delete
  10. Hi Linuxium, great work on this, but I keep getting a syntax error every time I try and run the script.

    Executing in the terminal as follows:

    sh isorespin.sh -i ubuntu-16.04.3-desktop-amd64.iso

    And it throws back with:

    isorespin.sh: 33: isorespin.sh: Syntax error: "(" unexpected

    Trying to run this in a live Ubuntu 16.04.3 USB with apt-get updated/upgraded. I've also tried a Linux Mint 18.2 distro and got the same result. What am I doing wrong?

    Cheers.

    ReplyDelete
    Replies
    1. It is a BourneAgainSH script so either run as './isorespin.sh' or 'bash isorespin.sh'.

      Delete
  11. doesn't install refind = doesnt even boot once you restart the device, cant install refind via refind-install because its running off the live usb so tries to install to usb which fails clearly, cant mount the mmc because it says its in use or busy,

    how exactly are you meant to get refind up and running? all of the tutorials on this site have had the refind installation information removed completely

    ReplyDelete
    Replies
    1. My documentation ('https://linuxiumcomau.blogspot.com.au/2017/06/customizing-ubuntu-isos-documentation.html') includes the section 'Installation on Intel Apollo Lake Devices' with a link which has the documentation on installation (see 'http://www.rodsbooks.com/refind/installing.html') so I didn't add anything else and nothing has been removed anything either. Do you think there is a need for additional instructions on installing 'rEFInd'?

      Delete
    2. well I can boot from the USB live distro and install, but when it reboots obviously it isn't even trying to use refind so its a black screen exactly the same as every other distro gives me. I've tried following the 'manual install' steps on rodsmiths site a million times over and it hasn't worked once.

      every single time it results in a black screen. exactly the same as if i never used respin at all. clearly there's a step missing to install refind instead of grub, that just isn't working as is....

      as it stands, the only way to do it on Apollo lake is dual boot windows 10 and that just destroys your mmc storage space so it's a complete waste of time when the respun iso doesn't even install refind

      frustrating to say the least and without refind being installed and the manual install simply not working it's not possible to do anything other than use the live image. which is a waste of time for permanent use. even using refind-install doesn't do the trick and upon reboot the same black screen with white cursor in the top left corner.

      if refind installed alongside the OS, it would be great... but no matter who I have asked and where I have looked for the past 3 weeks, not one single person has been able to show me exactly how I can have refind installed alongside the OS and have it work upon rebooting.

      I've formatted and reinstalled somewhere around 50 times so far and simply given up getting refind working after a reboot

      Delete
    3. Okay I'll put writing up a short tutorial on rEFInd installation on the TODO list.

      Delete
  12. I try to install on my wintab 10 baytrail and i receive error grub efi failed to install to /target/. Why is that?

    ReplyDelete
    Replies
    1. I'm assumming that you tried to install Lubuntu and you were not connected to the internet during the installation? Currently Lubuntu installation on 32-bit devices requires the internet. As per my documentation:

      My script solves this for Ubuntu ISOs as it will now add the required GRUB packages allowing the installation to successfully complete. However other Ubuntu ISOs including the Ubuntu flavours and distros based on Ubuntu (such as Linux Mint) etc. their installation without the internet on 32-bit devices is with varying success. In part it depends on the original ISO installation capability on 64-bit devices without the internet. For example as Lubuntu 17.04 cannot be installed on 64-bit devices without internet as the ISO does not contain '/pool' with required packages so likewise after respinning its installation on 32-bit devices is not possible.

      Lubuntu 17.10 Alpha 2 is interesting as it does contain '/pool' but doesn't contain the GRUB packages which is why the internet is still required.

      As a work-around you could try the following to respin the ISO as before but (hopefully) with the 32-bit GRUB packages. This assumes you have already downloaded my script and copied to '/usr/local/bin/isorespin.sh' and have also downloaded 'artful-desktop-amd64.iso' from 'http://cdimage.ubuntu.com/lubuntu/releases/17.10/alpha-2/' and copied to 'lubuntu-17.10-alpha2-desktop-amd64.iso' in your current directory:

      cp /usr/local/bin/isorespin.sh .
      sed -i '3101,3103c sudo mkdir -p iso-directory-structure/pool/main/g/grub2 && TRY_TO_ADD_32BIT_GRUB_PACKAGES' isorespin.sh
      ./isorespin.sh -i lubuntu-17.10-alpha2-desktop-amd64.iso --atom -k v4.13-rc2

      and then try installing again without the internet.

      Delete
    2. any solution on the sound driverS? I also tried with 2 different bluetooth headsets without success

      Delete
  13. i managed to fix it by reinstalling again ubuntu. Thanks a lot. Any solution with the sound does not working?

    ReplyDelete