Saturday, 3 June 2017

Compiling your own kernel using 'isorespin.sh'


Anyone who has tried to respin an Ubuntu ISO with the latest mainline kernel release candidate (v4.12-rc3) will have encountered a black screen as it fails to boot on Intel Atom devices. This is due to a new patch causing a regression and demonstrates the risks associated with using release candidate software in a production environment. The standard solution is to compile your own version of that kernel reverting the error-causing patch however not everyone feels confident in attempting this.

Because I've added functionality to my 'isorespin.sh' script that allows the addition/removal of files and directories together with the ability to execute commands and output their results it is relatively simple to use this new functionality to compile a kernel and create a custom ISO that uses it.

The two part process is to create a 'compile' ISO' which provides the kernel binaries that can then be installed as local packages in the second process which creates the 'install' ISO.



I've chosen as my first step to download the various source files to be used in the compilation. Whilst this can also be done as part of respinning there is no real benefit and certainly a disadvantage if performing multiple compilations.

Details of the required source files are documented on the Ubuntu Kernel Teams webpage (see http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.12-rc3). The mainline source can be downloaded using 'git' and the required Ubuntu patches can be downloaded using 'wget'.

I've also downloaded the 'revert' patch which will be applied as part of mainline kernel v4.12-rc4.



Next I have created some scripts which I will run as part of respinning a 'compile' ISO. They basically install the required dependencies for compiling the kernel, move the kernel source to where I want to compile it, patch the source ready for compiling, perform the compilation and finally remove the various source and patch files and directories once finished.

Some key points to note: The 'build-dep' uses the kernel from the respin ISO so it will vary depending on what ISO release is used (in my example it is 4.10.0-19-generic as used by the Ubuntu 17.04 ISO). I don't technically need to move the source from '/usr/local/bin' to '/usr/src' however this is my preferred location. The 'chmod' commands are required to fix the permissions and the removal of 'generic.inclusion.list' just ensures that all kernel modules are included in the 'linux-image' and no 'linux-image-extra' is created. As I am compiling on a quad-core CPU I have set 'DEB_BUILD_OPTIONS=parallel' to '4' with these options being documented in the Ubuntu wiki pages. Finally I remove the source once compiled in order to keep the size of the resultant 'compile' ISO down. You can 'output' the compiled source using the '-o' option when respinning however this would still require the source due to the precedence of how the options are run whilst respinning (i.e. '-c' options are all run before any '-o' options are processed by the 'isorespin.sh' script). Finally I can remove the 'remove' script as part of the 'remove' script without the need of a dedicated respin 'command' to perform the removal.



I can then run the 'isorespin.sh' command to patch and compile the source and output the compiled binaries as Debian packagess.



Looking at the command invocation in more detail you can see I've grouped some options purely for my readability of the logic I am using to respin the ISO. As I know what the binaries will be named and where they are located I can confidently include the '-o' option to output or save the files under their creation paths in the script work-directory 'isorespin' which is retained after running the script unlike earlier when respnning without this new option. Had I not know the exact details I could always boot the 'compile' ISO and extract the files manually.



Eventually the compilation and consequential respin will finish and the 'compile' ISO will be created together with the logfile and the binaries. If you are interested in knowing what is happening during the compilation a trick that can be used here is to run 'sudo tail -f isorespin/ics-chroot/root/.command.log' in another window as this will show the progress in real-time.



The logfile is rather large.



Any errors encountered during the compilation will be written to the logfile as will any successful outcomes.



Prior to creating the 'install' ISO the files produced by the compilation must be preserved to prevent being overwritten.



To create the 'install' ISO the script is then re-run and a new ISO will be respun installing the previously produced binaries as 'local-packages'.



Once finished the new ISO can be written to a USB using 'dd'.



Booting this ISO which uses your new v4.12-rc3 kernel now works on your Intel Atom device.

Another key advantage of this approach is that any tablet user who requires specific non-mainline patches can relatively easily include additional patches and compile their own customized kernel for their device.

If you find this useful then please donate using the following link http://goo.gl/nXWSGf as everything helps with development costs.

6 comments:

  1. I have not tested RC3, but RC2 and RC1 will shutdown my screen back-light on my tablet on boot, I notice that the issue is known for some time (ex https://bugs.freedesktop.org/show_bug.cgi?id=96923 or https://bugs.freedesktop.org/show_bug.cgi?id=96571) but fix not available neither in current kernel, maybe RC4 ....

    It need to pass at least some time to have a functional tablet (Trekstor Wintron 8.0), main issue being screen and touch response regression with latest kernel releases...

    ReplyDelete
  2. v4.12 RC4fixes the backlight issue (at least on my Asus E200HA).
    I still experience the old problems:
    * (almost) broken microSD card reader
    * no internal sound (conexant 2702)
    * flakey Bluetooth

    ReplyDelete
  3. I'm new to all this respinning etc, can you do a newbie guide on how to do this for say ubuntu on compute stick?

    ReplyDelete
    Replies
    1. See my latest post 'http://linuxiumcomau.blogspot.com.au/2017/06/interim-ubuntu-1704-iso-with-mainline.html' for an example if the other posts cannot help.

      Delete
    2. That worked a treat. I will throw you a few $ to show appreciation for your work. A simplified guide would still be appreciated, so I can learn to do this myself.

      Delete
    3. Thanks. I'll write an updated guide.

      Delete