Wednesday, 21 June 2017

Customizing Ubuntu ISOs: Documentation and examples of how to use 'isorespin.sh'


Normally Linux distribution ISOs work perfectly when written to a USB for booting as a 'Live USB' allowing both usage and installation of the distro. Unfortunately with the introduction of Intel Atom based mini PCs and tablets the issue of requiring a 32-bit bootloader to boot a 64-bit OS arose. Further complications were caused by the lack of 'mainline' support for HDMI audio and wifi/bluetooth resulting in the need for custom kernels. Although some Linux distros can work OOTB I found that for Ubuntu it wasn't that simple.

To solve the limitations of using a standard ISO I developed a script that allows an ISO to be respun and customized ("remastered") to create a new ISO. Being a script which you run locally it means you can both see and control how your ISO is respun. I wanted flexibility in the ability to:
  • Add a GRUB 32-bit bootloader to allow ISOs to boot on the many Intel Atom devices limited by their BIOS or to add the rEFInd boot manager to allow booting on Intel Apollo Lake devices.
  • Upgrade the kernel to the latest or to a specific version to benefit from recent patch functionality e.g. for audio and wifi/bluetooth.
  • Add (or remove) repositories, software packages, scripts and files to allow the installation of firmware and favourite programs.
  • Preseed or perform set-up commands and scripts and also change system defaults.
  • Add persistence so that the ISO can retain data, installed software packages and settings across reboots.
The result is my 'isorespin.sh' script which works with all official desktop 64-bit Ubuntu (http://releases.ubuntu.com) and Ubuntu flavoured ISOs (https://www.ubuntu.com/download/ubuntu-flavours), Linux Mint ISOs (https://www.linuxmint.com/download.php), KDE neon ISOs (https://neon.kde.org/download) and elementary OS ISOs (https://elementary.io).

It can be run using either the shell CLI (command line interface) in a the terminal or using its GUI (graphical user interface). Not only will it generate a new ISO but it will also produce a log file which includes the options used when respinning the ISO and serves to document the respun ISO.

Although it is expected to run the script on a Linux machine it also works on a Linux virtual machine on Windows (see below for further details). You will need certain packages like 'squashfs-tools' and 'xorriso' installed (use 'sudo apt-get install -y squashfs-tools xorriso' to install them) plus 'zenity' if you want to use the GUI. As different Linux distros have different packages installed by default the script will first check and notify you if any other packages are required. You will also need at least 10 GB of free space but this can be on external storage (e.g. a USB) as it can be specified as a location different to where the script is run from. For some options (such as updating the kernel) you will need a working internet connection as the script needs to download software. Also the script needs to be executable which can be achieved using the 'chmod' command ('sudo chmod 755 isorespin.sh'). Finally as the script runs some commands (e.g. mount/umount) which require root access using the 'sudo' command you will initially be prompted for your password and must already have 'sudo' privileges.


Invocation

Running the script is really quite simple even though there are quite a lot of features or options making it look rather complex.

Usage: /usr/local/bin/isorespin.sh [ -h | -v ]
       /usr/local/bin/isorespin.sh -i <ISO> [ [ -u | -k <kernel> ] | -r "<repo> ... " | -p "<pkg> ... " | -l "<pkg.deb> ... " | -f "<file> | <directory> ... " | [ -s <size>MB | GB ] | [ -b GRUB | rEFInd ] ...
       /usr/local/bin/isorespin.sh ... -w <directory> | -d "<pkg> ... " | -e "<pkg> ... " | -c "<cmd> ... " | -o "<file> | <directory> ... " | -g "" | "<kernel boot parameter> ... " ]


Respinng an ISO is controlled by either running the script with flags and arguments (CLI) or selection options and values (GUI) and briefly these are as follows:

-h or --help will display the 'usage' message. 
-v or --version displays the version of the script. 
-i or --iso must be specified and is the name (including the full or relative path) of the ISO to be respun. 
-u or --update will update the kernel to the latest Ubuntu Kernel Team kernel builds located at http://kernel.ubuntu.com/~kernel-ppa/mainline
-k or --kernel replaces the kernel with the version specified by the argument or value which can be any of the Ubuntu Kernel Team kernel builds located at http://kernel.ubuntu.com/~kernel-ppa/mainline and is passed as the directory or folder name without the trailing '/' (e.g. '--kernel v4.11-rc7'). 
-r or --repository adds the specified repository to the respun ISO. 
-p or --package will install a single package or a set of packages (if enclosed in quotes) that are available to the ISO and would typically be installed using the 'apt-get install <package>' command. This option is very useful in preseeding your ISO with packages you always normally install immediately after booting and will help resolve the issue of the ISO running out of space as a result of multiple installations. For example I typically include "ssh openssh-server inxi" when respinning an ISO for personal use.
-l or --local allows local Debian binary packages to be installed within the respun ISO. Care must be taken in specifying the order of multiple packages to preserve any dependencies and of course all dependencies must be met for the packages to be successfully installed. 
-f or --file will copy either the specified files or directories to the respun ISO under '/usr/local/bin'. Although intended to allow local binaries or shell scripts to be saved on the respun ISO there is no limitation on the actual type of file that can be added. 
-s or --storage adds a specified amount of persistent storage to the respun ISO. A size must be specified although there is some flexibility on how much is allocated. The minimum size is 128MB which keeps the size of the ISO down but means that once a USB is created the persistence partition needs to be manually resized for normal usage. Alternatively a larger size can be specified to avoid the immediate need to resize as this can always be performed later. It is recommended to use a realistic amount somewhere between 1GB and 2GB to maintain the balance between a practical but usable ISO. When creating a respun ISO with persistence the rEFInd boot manager will automatically be selected. When using the GUI the persistence partition size can be defined using the slider which limits the size between 128MB and 2048MB.
-b or --boot will add either the GRUB boot loader (which is added by default) or the rEFInd boot manager which currently is required for booting on a lot of the current Intel Apollo Lake devices. 
-w or --work-directory is used to specify where the ISO will be respun. This is primarily for overcoming the lack of space associated with mini PCs and allows the respinning to be run on temporarily connected external storage media such as a drive or USB. Once the storage has been connected and mounted it can be passed as a parameter. Likewise if a directory is then created on the mounted storage this can be passed as the argument or value for this option to ensure there is sufficient space (i.e. at least 10 GB) for the script to run.
-d or --download will download Debian binary packages from the respun ISO using the 'apt-get download' command and save them under '/usr/src'. If combined with the '-o' option the Debian binary package will be accessible after respinning the ISO. 
-e or --erase will remove Debian binary packages from the respun ISO using the 'apt-get purge' command. 
-c or --command will execute the supplied argument as a 'bash' command. Care must be taken when embedding single or double quotes within the argument as they are subject to interpretation. For more complex commands it is often both advisable and easier to write a simple shell script and use the '-c' option to execute it as part of respinning. 
-o or --output will save either the specified files or directories to an output directory called 'isorespin'. This is very useful when wanting to extract files or packages that have been created or downloaded as part of respinning the ISO. 
-g or --grub will add the specified kernel boot parameters both to the GRUB/rEFInd boot menu and to the default GRUB settings or if an empty ("") argument is specified then the the ISO's initial kernel boot parameters will be removed.

The script can either be run as a local script or it can be installed by copying the script to '/usr/local/bin' where it will be accessible for all users after the script has been made executable using the 'chmod' command ('sudo chmod 755 isorespin.sh'). Depending on the options and their complexity the respinning will take quite a few minutes and the script provides updates on its progress whilst running. The respun ISO is created with a new name derived from the original ISO name and includes indicators of the options used like 'persistence' or the kernel version for example with full details being written to the log file.

Once the ISO has been respun it can be written to a USB using the standard 'dd' command.

The examples below illustrate the various options through scenarios and provide a basic tutorial to using the script. They are based on the script being installed to '/usr/local/bin':


The script's help is displayed using the command:

isorespin.sh -h


The version of the script can either be displayed by entering:

isorespin.sh -v


or it can be seen from the top of the script's 'main' menu:


or 'advanced' menu:


The script can be run using the CLI by specifying options with arguments in any order. If the script is started without any parameters it first checks to see if it can run the GUI and if so starts with the main selection menu where all required options should be selected. As seen above the GUI uses two menus with the first being a main menu of most frequently used options and an optional additional menu of the more advance options. For each option selected, a sub-menu will appear to allow the choice or value to be entered.


Respinning an ISO

The simplest usage is to respin an ISO which will automatically add the GRUB 32-bit bootloader allowing the respun ISO to boot on Intel Atom devices e.g.:

isorespin.sh -i ubuntu-17.04-desktop-amd64.iso

or by invoking the GUI:





Upgrading the kernel

To respin the ISO with the latest kernel the command would be:

 isorespin.sh -i ubuntu-17.04-desktop-amd64.iso -u

or



If a specific kernel was required for example 'v4.11.6' then the '-k' option would be used:

isorespin.sh -i ubuntu-17.04-desktop-amd64.iso -k v4.11.6

or



Adding local packages and running scripts

A more complex scenario of respinning an ISO is when you want to add packages and run scripts to fix functionality like wifi/bt and audio.

Due to current limitations of the official ISOs I've previously created scripts to add wifi/bt firmware and also add the ALSA UCM audio files:

rtl8723bs_4.12.0_amd64.deb
rtl8723bt_4.12.0_amd64.deb
linuxium-install-broadcom-drivers.sh
linuxium-install-UCM-files.sh

These can be incorporated when respinning an ISO. For the scripts you first add the file using the '-f' option and then execute the script using the '-c' option.

When I wrote the above scripts I originally intended them to be run post installation rather than as part of respinning an ISO. As a result they produce warnings and also need to be executed in the correct directory. Rather than writing a complex '-c' parameter to run them correctly it is simpler to write a further script (which I call a 'wrapper') which controls how the script is run. These 'wrappers' can then be included as part of respinning an ISO:

wrapper-linuxium-install-broadcom-drivers.sh
wrapper-linuxium-install-UCM-files.sh

For example the following command will respin the Ubuntu 17.04 ISO and upgrade the kernel to the latest version (which includes the RTL8723BS driver) installs the required RTL8723BS wifi/bt firmware and both adds and runs a script to include the UCM files for audio:

isorespin.sh -i ubuntu-17.04-desktop-amd64.iso -u -l rtl8723bs_4.12.0_amd64.deb
-f linuxium-install-UCM-files.sh -f wrapper-linuxium-install-UCM-files.sh
-c wrapper-linuxium-install-UCM-files.sh


While the script is running progress updates are displayed:


until it finally completes:


producing a log file which shows how the script was run:



A similar example but respinning the daily snapshot Ubuntu 17.10 (Artful) ISO and using the GUI starts by selecting the required options:


and then entering the values starting with the ISO. When selecting the ISO a 'file manager' styled window will be presented but only files ending with a '.iso' suffix will be displayed:


For the kernel upgrade option the choice is to upgrade to the latest mainline version:


Local packages are selected using the 'file manager' which only displays '.deb' files. To allow multiple entries you will be asked after each selection is made whether a further addition is required:



Next the two scripts are added using the 'file manager':





And finally the 'advanced' menu is displayed and after selecting the option to add a command:


the command is entered as a text string:


Now that all the selected options have been entered a confirmation screen is be displayed which shows the full command as it would look if the script had been run manually:


Having selected 'yes' the script runs and displays its progress:





until it finishes:


Similarly a log file is produced that shows how the script was run:



For a more complex example of using the script see Compiling your own kernel using 'isorespin.sh'.


Persistence

Adding persistence to an ISO is useful as it means that you can retain your data, installed software packages and settings between reboots. Persistence can be added as part of any ISO respinning simply by adding the '-s' option and defining the size of the storage to be permanently allocated in the ISO. As previously mentioned there is a balance between making it small and therefore resulting in a small ISO that requires resizing once booted (see below) verses a larger size and a corresponding larger ISO.

The following example respins the standard Ubuntu 17.04 ISO with the latest kernel and create a 2 GB persistence partition:

isorespin.sh -i ubuntu-17.04-desktop-amd64.iso -u -s 2 GB

Using the GUI it is created by:








As can be seen from looking at the size of the resulting ISO that it is large at around 3.8 GB but not too large to be totally impractical.


Kernel boot parameters

The following is an example showing both the removal of the default kernel boot parameters (quiet splash) and the inclusion of two new kernel boot parameters:

isorespin.sh -i ubuntu-17.04-desktop-amd64.iso -g "" -g "i915.fastboot=1 fbcon=rotate:1"

The default kernel boot parameters are first removed by selecting the delete option:


and the new kernel boot parameters are then added:


as text strings:



Once the script is running it will update the kernel boot parameters:


with the exact details being recorded in the log file:



Creating a LiveUSB

Once your respun ISO has been created you can write it to a USB using 'dd'. For example if your USB is '/dev/sdb' then enter:

dd if=linuxium-v4.12-rc6-ubuntu-17.04-desktop-amd64.iso of=/dev/sdb bs=4M

Make sure you select the correct device for the USB and always check first using commands like 'df', 'blkid' or 'lsblk' to confirm. ISOs can also be written to a USB using 'Rufus' on Windows if preferred.


Booting your respun ISO


Having created the USB with your respun ISO the following information may be useful when booting and running.

To boot your device select the USB's UEFI partition from the device's boot menu:


Normally you will then see a standard GRUB menu:



If the 'rEFInd' boot manger was installed (either through selection or automatically due to persistence) the following screen will be displayed for 64-bit bootloader devices:



otherwise for 32-bit bootloader devices the normal the GRUB menu screen is displayed and after selection a black screen with the following message is briefly displayed:

error: no suitable video mode found.
Booting in blind mode

before the normal Ubuntu splash screen appears and the device fully boots. This 'error' can safely be ignored.

Wifi issues

If your device uses a Broadcom wifi chip and wifi isn't working even after adding the firmware using my script then it may be missing an SDIO file. By running the command 'dmesg' immediately after booting and looking at the output if you see messages showing that the wifi firmware 'txt' file is missing it is possible to use the copy from the Windows driver that is stored in NVRAM. The file should look something like '/sys/firmware/efi/efivars/nvram-74b00bd9-805a-4d61-b51f-43268123d113' and you actually copy it using 'cat /sys/firmware/efi/efivars/nvram-74b00bd9-805a-4d61-b51f-43268123d113 > /lib/firmware/brcm/brcm/brcmfmac43241b4-sdio.txt' substituting the actual NVRAM filename on your device and the correct *sdio.txt name for your driver.

Bluetooth issues

On booting if bluetooth fails to work automatically and the firmware was installed using one of my scripts then the service can be manually started with the command:

sudo systemctl start rtl8723bsbt.service

or

sudo systemctl start brcmbt.service

depending on device's chip. If bluetooth is always required then the command could be included in '/etc/rc.local' to avoid this step. Note that this uses 'systemd' so earlier ISO releases without 'systemd' cannot use my scripts to provide bluetooth.


Resizing the persistence partition

Once booted use the 'gparted' command to resize the persistence partition. You will initially be prompted to fix the GPT to use all the available space so click 'Fix'


and then continue with the resizing:



NVRAM sync issues

Sometimes after installing your device only boots to a black screen.  Typically this happens when the NVRAM and ESP are out of sync. This can be easily fixed by performing the following:

1. Start with the device switched off.
2. Connect your LiveUSB.
3. Power on the device and press the relevant function key or key sequence to boot from the LiveUSB.
4. Open a terminal windows and enter:
type efibootmgr 
5. If the 'efibootmgr' is not installed enter:
sudo apt-get update && sudo apt-get install -y efibootmgr
6. Next remove any boot entries that may have been created through the earlier installs by entering:

      for BOOTENTRY in 0 1 2 3 4 5 6 7 8 9
      do
          sudo efibootmgr -b ${BOOTENTRY} -B
      done

7. Now create an Ubuntu boot entry by entering:
sudo efibootmgr -c -d /dev/mmcblk0 -p 1 -l \\EFI\\ubuntu\\grubx64.efi -L Ubuntu
8. Remove the LiveUSB and reboot from the newly created Ubuntu entry.


Note: If you have a dual boot installation then the ESP partition may be the second partition so modify the '-p 1' in the command above to '-p 2' or to whatever the ESP partition is. Likewise for some devices the eMMC is '/dev/mmcblk1' and not '/dev/mmcblk0' so again modify the command as appropriate. If your device uses a 32-bootloader then use 'bootia32.efi' rather than 'grubx64.efi' in the command. It also sometimes doesn't work first time and requires all the steps to be repeated which should then fix the issue.


Additional information regarding the script

Minimizing repeated downloads

The rEFInd boot manager (http://www.rodsbooks.com/refind) can be downloaded from https://sourceforge.net/projects/refind/files into the same directory as the script is run from to prevent unnecessary repeated downloads if this is a concern and will make processing quicker.

Avoiding new windows opening when respinning

To prevent the file manager opening a window each time the script mounts a temporary file system I have run the command 'gsettings set org.gnome.desktop.media-handling automount-open false' first on my system.

Lock file

The script creates a lock file '.isorespin.sh.lock' in the directory where it is run from in order to ensure exclusive access to the run-time environment. If the script or machine crashes this lock file may need to be removed in order to rerun the script and you will be prompted should this occur. However it is not recommended to run the script in parallel from multiple directories as it is both CPU and I/O intensive and severe performance degradation would occur.

Permissions

If when running the script you get the error:

bash: ./isorespin.sh: Permission denied

it means that the script needs to be made executable which can be achieved using the 'chmod' command ('sudo chmod 755 isorespin.sh').

Running the script on Windows

First download and install Oracle's VirtualBox on Windows and then create a Linux VM from a standard Ubuntu 64-bit desktop ISO and install to a VDI of fixed size (minimum 20GB). Start the new Ubuntu VM and install 'Oracle VM VirtualBox Extension Pack'. You can then download my 'isorespin.sh' script to respin the Ubuntu-based ISO of choice. By installing the extension pack you will also be able to write the respun ISO to a USB using 'dd' from within the Ubuntu VM. Unfortunately 'Bash on Ubuntu on Windows' using 'Windows Subsystem for Linux' does not support 'loop' devices which are required by my script so an Ubuntu VM is the only working alternative at this time.

Running the script on Arch Linux

Running the script on Arch Linux requires a modified PATH. The simplest way is to run the script as follows:

PATH=/usr/sbin:/sbin:/bin:$PATH ./isorespin.sh

Additionally to install the required packages it may be necessary to run:

sudo pacman -S cdrkit bc libisoburn squashfs-tools dosfstools

or similar depending on which packages are missing from you environment.


Miscellaneous issues not directly related to the script

The Linux mainline kernel

It is worth noting that certain releases of the mainline kernel include specific core functionality:

v4.9 includes the eMMC v5.0 driver required for accessing the internal storage in many devices.
v4.11 includes the HDMI audio driver.
v4.12 includes the RTL8723BS driver although you will still need the wifi/bt firmware.

Kernel freezes

Well known and well documented without a permanent fix or so it seems. Kernels after v3.16 seem to include something that caused them to randomly freeze on Intel Atom devices and then the whole system hangs. Subsequently an accepted workaround was recommended to limit the processor (CPU) to a certain power state or 'C-state' and if freezes are encountered then to pass 'intel_idle.max_cstate=1' as a boot parameter. Later kernels have included patches and the current state seems to be reliable enough to ignore the workaround. If however freezes still occur the easiest way to implement the workaround on an installed system is to edit the file '/boot/grub/grub.cfg' and modify to look like:


GRUB_CMDLINE_LINUX_DEFAULT="intel_idle.max_cstate=1"

Alternatively the following command can be entered:
sudo sed -i 's/\(GRUB_CMDLINE_LINUX=\)""/\1"intel_idle.max_cstate=1"/' /etc/default/grub
followed by:
sudo update-grub
and then a reboot as the change only needs to be made once typically following installation.

Micro SD card issues

Whilst micro SD cards have worked on Intel Atom Bay Trail devices it is only since the v4.7.2 kernel that Intel Atom Cherry Trail devices have had any form of success.

The typical error encountered is:

        mmc1: error -110 whilst initialising SD card
        mmc1: card never left busy state

and although most Class 10 and below cards from any manufacturer except Sandisk work now without problem some UHS cards still fail. I've found Samsung UHS Speed Class 3 and UHS Speed Class 1 work whereas similar Sandisk cards fail crashing the device sometimes when inserting or removing the card.

No sound from headphones or internal speakers

It may be necessary to first use the speaker controls to select the audio output before sound works. For audio on Lubuntu this will also require installing the package 'pavucontrol':

sudo apt-get install -y pavucontrol

and then using it to select the audio output e.g. HDMI. Unfortunately some devices still require additional audio drivers or specific device quirks that have not been incorporated into the mainline kernel so sound may only work over HDMI and not through internal speakers or via the headphone jack even though the UCM files have been added. This is part of ongoing Linux development and future kernel releases hopefully will include the drivers required for currently non-working devices.

Installing without an internet connection on devices with a 32-bit boot loader

Unfortunately the standard Ubuntu ISO requires an internet connection when installing on devices with a 32-bit boot loader otherwise the following error is encountered:
The 'grub-efi-ia32' package failed to install into /target/. Without the GRUB boot loader, the installed system will not boot.
My script solves this for Ubuntu ISOs as it will now add the required GRUB packages allowing the installation to successfully complete. However other Ubuntu ISOs including the Ubuntu flavours and distros based on Ubuntu (such as Linux Mint) etc. their installation without the internet on 32-bit devices is with varying success. In part it depends on the original ISO installation capability on 64-bit devices without the internet. For example as Lubuntu 17.04 cannot be installed on 64-bit devices without internet as the ISO does not contain '/pool' with required packages so likewise after respinning its installation on 32-bit devices is not possible. However Lubuntu 16.04.2 can be installed on both 64-bit and 32-bit devices after respinning (see http://cdimage.ubuntu.com/lubuntu/releases/16.04.2/release/lubuntu-16.04.2-desktop-amd64.list and http://cdimage.ubuntu.com/lubuntu/releases/17.04/release/lubuntu-17.04-desktop-amd64.list for detailed package information). Also for non-Ubuntu ISOs it depends on the availability of archived package versions as some ISOs are created with earlier GRUB package versions to those currently available. For example with Elementary OS the earlier packages are not available and it is not known where (or if) an archive copy is maintained so installing without the internet on 32-bit devices is not possible.

Suspend/Hibernate/Sleep issues

Suspend in general has been problematic on Intel Atom devices and while new patches are continually being applied to the mainline kernel it is currently not working or incorrectly working on a number of devices.

Brightness issues

Unfortunately as I don't have a tablet I cannot advise on the various screen related issues some users experience using Ubuntu. The best command appears to be 'xrandr' although its use is device specific. Many people have commented in previous posts with their solutions and I've included some of these below.

Most brightness suggestions include:

xrandr --output [device_name] --brightness floating_pvalue (0 <= x <= 1.0)

where you can determine your device by entering:

xrandr -q | grep "connected"

with the primary one being your device.

For example:

xrandr --output DSI-1 --brightness .${BRIGHTNESS}

where the variable BRIGHTNESS can take a value between 3 and 9.

Rotation issues

Some examples that users have posted include:

xrandr --output DSI-1 --rotate right
xinput set-prop <Device Touchscreen> 'Coordinate Transformation Matrix' 0 1 0 -1 0 1 0 0 1

and

xrandr -o right
xinput set-prop 'Goodix Capacitive TouchScreen' 'Coordinate Transformation Matrix' 0 1 0 -1 0 1 0 0 1

Also others have reported that screen rotation during boot works with 'fbcon=rotate:1'.

Touchscreen issues

These seem to be related to rotation issues and another example of a solution includes:

xinput set-prop "Silead GSLx680 Touchscreen" "Coordinate Transformation Matrix" 1 0 0 0 -1 1 0 0 1

which can also be set when matching InputClass section in 'xorg.conf' with:

Option "TransformationMatrix" "1 0 0 0 -1 1 0 0 1"

Installation on Intel Apollo Lake Devices

On hardware requiring the rEFInd boot manager to boot such as Intel Apollo Lake devices you will need to install the rEFInd boot manager manually (see http://www.rodsbooks.com/refind) as the standard installation only includes the installation of GRUB.


Reporting issues

Comments are welcome but when discussing an issue please include the name of your device and use 'pastebinit' or similar to post a URL to a copy of the output from 'dmesg' (which is basically a log of kernel messages).

To use the 'pastebinit' command (see https://help.ubuntu.com/community/Pastebinit) first install the command by entering 'sudo apt-get install -y pastebinit' and then to share the kernel messages log enter 'dmesg | pastebinit'. This will paste a copy of the output from 'dmesg' to Ubuntu's Pastebin and provide a URL to access it.

So when reporting an issue please include:
  • Outline of issue
  • Name of device
  • URL from pastebin (e.g. http://paste.ubuntu.com/23646692/)


Donate

Please donate if you find the script useful using the following link http://goo.gl/nXWSGf as everything helps with development costs.

86 comments:

  1. -script does not generate iso

    -im using intel core i5 pc , 8 gb ram, 100gb hd free space, opensuse 13.2 x64.

    https://pastebin.com/4vGyXxy6

    Please help,
    Thanks in advance.

    ReplyDelete
    Replies
    1. openSUSE 13.2 is EOL and as you seem to be hitting a known 'sudo' bug I'd recommend upgrading before trying my 'isorespin.sh' script again.

      Delete
    2. better downloaded your .iso https://goo.gl/hZ4L1x with kernel 4.12rc6 and it worked! on my HP Stream 7 tablet, wireless worked, sound, bluetooth not tested yet, brightness works using xrandr.

      Thanks!

      Delete
  2. You state "If bluetooth is always required then the command could be included in '/etc/rc.local' to avoid this step."

    I think the better solution is to simply use:

    $ sudo systemctl enable ...

    ReplyDelete
    Replies
    1. 'enable' only links the service files and you need 'start' to actually start the service.

      Delete
  3. Your Script worked perfectly using Ubuntu Unity 17.04 on a 4-64 LattePanda. Wireless works Fine. Even Bluetooth works! I tried it with my Sony Bluetooth speaker and the speaker worked perfectly. I incorporated the 4.12 rc6 Kernel as well.

    Known issues: Ghosting Cursors in Firefox. Looks like an old PacMan game. No functional issues but you have to remember where the real cursor is as it will vanish. All the other ghosts are like mouse trails.

    I tried the XUbuntu and never could get System Tray to display. (You get a blank screen.) Right click gives you your full menu but no applications dealing with the screen will launch. Firefox, Terminal, Updates, and File Manager all work normally.

    Is there a bug affecting XFCE when run on atom processors? The processor is an Atom Processor is a z83XX unit. If this is of interest, let me know I can be more specific.

    ReplyDelete
    Replies
    1. Do you get the same problem with XFCE using 16.04.2 vs 17.04 on the LattePanda device?

      Delete
  4. I neglected to mention, I ran the Broadcom drivers Script post install.

    I haven't tried Samba yet, but that install and configure will be this week.

    ReplyDelete
  5. Does not work :(

    isorespin.sh: '/home/brian/ubuntu-16.04.1-server-amd64.iso' must be an Ubuntu (or Ubuntu flavour), Linux Mint, elementary, or neon desktop ISO.

    Really wish that ISO was still available!!

    ReplyDelete
    Replies
    1. As per above the script works with all official desktop 64-bit Ubuntu ISOs.

      Delete
  6. Does not work on Desktop ISO files either:

    isorespin.sh: '/home/brian/ubuntu-16.04.1-desktop-amd64.iso' must be an Ubuntu (or Ubuntu flavour), Linux Mint, elementary, or neon desktop ISO.

    ReplyDelete
    Replies
    1. It does works with 'ubuntu-16.04.1-desktop-amd64.iso' so can you check your ISO? An 'ls -l' should show a size of '1513308160' and a 'md5sum' should give '17643c29e3c4609818f26becf76d29a3'.

      If they are different your ISO is likely to be corrupt and you should download it again.

      Next check the 'isorespin.sh' script. An 'ls -l' should give '3937300' and an 'md5sum' give '5e5b86cdb69fdcdd7a1edf969286eb20'.

      Finally what OS are you running the script on? What do you get if you run the command 'cat /etc/lsb-release' or from running 'lsb-release -a'?

      Delete
    2. This error was caused by a missing package. I've just moved the package check to the start so the script will tell you immediately now.

      Delete
  7. Good morning,

    I have attempted to use your script but it gives multiple errors, before giving a final error of "Cannot find mainline kernel." and quitting. (Yes, it is connected to the internet.)

    Here is the environment: newly installed Ubuntu 16.04 on VMware (installed using express install as that is the default when you select a Ubuntu ISO). Network in bridged mode

    Executed sudo apt-get update; sudo apt-get upgrade; sudo apt-get install -y squashfs-tools xorriso zenity;

    Downloaded isorepin.sh (ver 7.1.2), as well as linuxium-isntall-UCM-files.sh and wrapper-linuxium-install-UCM-files.sh and rtl8723bs_4.12.0_amd64.deb - all in /iso as well as the same ISO I just built this VM from.

    Running from /iso which has been chmod 777.

    Running the following:
    sudo ./isorespin.sh -i ubuntu-16.04.2-desktop-amd64.iso -u -l rtl8723bs_4.12.0_amd64.deb -f linuxium-install-UCM-files.sh -f wrapper-linuxium-install-UCM-files.sh -c wrapper-linuxium-install-UCM-files.sh -s 2GB

    Yields multiple errors:
    Extracting ISO ...
    rsync: read errors mapping "/iso/isorespin/mnt/EFI/BOOT/BOOTx64.EFI": Input/output error (5)
    rsync: read errors mapping "/iso/isorespin/mnt/EFI/BOOT/grubx64.efi": Input/output error (5)
    rsync: read errors mapping "/iso/isorespin/mnt/boot/grub/efi.img": Input/output error (5)
    [MANY more of these rsync errors]

    Followed by a bunch of these failed verification errors:
    ERROR: boot/grub/efi.img failed verification -- update discarded.
    ERROR: boot/grub/font.pf2 failed verification -- update discarded.
    ERROR: boot/grub/grub.cfg failed verification -- update discarded.
    [MANY more follow]

    Those failed verification errors / update discarded are followed by:
    rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.1]
    Read on filesystem failed because Input/output error
    Read on filesystem failed because Input/output error
    Can't find a SQUASHFS superblock on mnt/casper/filesystem.squashfs
    mv: cannot stat 'squashfs-root': No such file or directory
    Extracting isorespin files ...
    Updating bootloader/bootmanager ...
    mv: cannot stat 'iso-chroot/dev': No such file or directory
    cp: cannot stat 'iso-chroot/dev.linuxium': No such file or directory
    mv: cannot stat 'iso-chroot/run': No such file or directory
    cp: cannot stat 'iso-chroot/run.linuxium': No such file or directory
    mv: cannot stat 'iso-chroot/tmp': No such file or directory
    cp: cannot stat 'iso-chroot/tmp.linuxium': No such file or directory
    cp: cannot create regular file 'iso-chroot/etc/': No such file or directory
    mv: cannot stat 'iso-chroot/etc/apt/sources.list': No such file or directory
    sed: can't read iso-chroot/etc/apt/sources.list.linuxium: No such file or directory
    tee: iso-chroot/etc/apt/sources.list: No such file or directory
    chroot: cannot change root directory to 'iso-chroot': No such file or directory
    [There's a few more cannot stat errors]

    Finally:
    Fetching mainline kernel packages ...
    Installing mainline kernel packages ...
    cp: target 'iso-chroot/usr/src/' is not a directory
    cp: cannot create regular file 'iso-chroot/usr/src/': No such file or directory
    mv: cannot stat 'iso-chroot/dev': No such file or directory
    cp: cannot stat 'iso-chroot/dev.linuxium': No such file or directory
    ... (removed some to get below 4096 chars)
    mount: mount point iso-chroot/dev does not exist
    umount: iso-chroot/dev: mountpoint not found
    mv: cannot stat 'iso-chroot/dev.linuxium': No such file or directory
    mv: cannot stat 'iso-chroot/run.linuxium': No such file or directory
    mv: cannot stat 'iso-chroot/tmp.linuxium': No such file or directory
    ./isorespin.sh: Cannot find mainline kernel.

    While it runs, it does create (temporarily) an isorespin directory.

    It looks to me like an initial error with the rsync call is causing the rest of the errors... but I'm not sure how to resolve the rsync error.

    Any advice?

    Thanks!
    -Eric

    ReplyDelete
    Replies
    1. In case you ask:

      linux-user@ubuntu:/iso$ cat /etc/lsb-release
      DISTRIB_ID=Ubuntu
      DISTRIB_RELEASE=16.04
      DISTRIB_CODENAME=xenial
      DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS"

      linux-user@ubuntu:/iso$ ls -l ubuntu-16.04.2-desktop-amd64.iso
      -rw-rw-r-- 1 linux-user linux-user 1167360 Jun 30 02:31 ubuntu-16.04.2-desktop-amd64.iso

      linux-user@ubuntu:/iso$ md5sum ubuntu-16.04.2-desktop-amd64.iso
      c2db7847895289963539ce4cb9822f31 ubuntu-16.04.2-desktop-amd64.iso

      linux-user@ubuntu:/iso$ ls -l isorespin.sh
      -rwxrwxr-x 1 linux-user linux-user 430080 Jun 30 02:41 isorespin.sh

      linux-user@ubuntu:/iso$ md5sum isorespin.sh
      4354f50ced7c5429518e8ae57be52996 isorespin.sh


      Delete
    2. FYI, I'm trying to get Ubuntu working on a Cherry Trail device. Thanks again.

      Delete
    3. Can you download the script again and retry as I think you might have a package missing. I've just moved the package check to the start so the script will tell you immediately now.

      Delete
    4. So that didn't fix it... BUT, I think I've identified the issue... and I should have noticed it earlier. When I copied the ISO over from my host, VMware tools did NOT actually copy the entire ISO to the Ubuntu guest (and it actually didn't copy the entirety of your script either). Making sure the full copy of the ISO makes it to the guest and trying again.

      Delete
    5. The 'md5sum' for 'isorespin.sh' should be '8bbfd21b46c0ebbcc05973c7b462988a' if it helps.

      Delete
    6. md5 matches this time... and I'm about 15 minutes into the execution without errors... so far, so good. Thanks!

      Delete
  8. /usr/local/bin/isorespin.sh: ISO '/home/chris/Documents/ubuntu-17.04-desktop-amd64.iso' is not a 64-bit (amd64 or x86_64) ISO.

    Both GUI and CLI have the same output(of course).
    Running on a fresh and updated install of Ubuntu 64bit (on VM).

    On the CLI it also outputs "grep mnt/.disk./iso input/output error"
    It has been run with sudo, in case it cannot mount.

    What gives? :/

    ReplyDelete
    Replies
    1. There was a small window yesterday when having modified the script to support the new '-g' option I uploaded a version which checked for required packages in the wrong place. So if you ran the script without the prerequisite packages it borked rather than stop with a message as it should do. Testing didn't pick this up as it worked if the packages where installed. I have corrected this by uploading a fixed version and now if you download and run the script it will work both correctly and as before. Now and as per the documentation 'You will need certain packages like 'squashfs-tools' and 'xorriso' installed (use 'sudo apt-get install -y squashfs-tools xorriso' to install them) plus 'zenity' if you want to use the GUI. As different Linux distros have different packages installed by default the script will first check and notify you if any other packages are required'.

      Delete
    2. Hello again and thank you for your reply.

      I had already installed these two packages (zenity was already installed), I read the guide twice :D

      I will try the updated script and I'll also try a physical Linux machine. I've encountered many strange issues with VMs in the past.

      Thank you for your time :)

      Delete
  9. Olla again any idea what i did wrong here

    [code]
    Script './isorespin.sh' called with '-i /home/hp/Downloads/kodibuntu-14-0rc-64-bit-multi-ubu.iso -k 4.12.0-041200rc6 -p "" -p "/home/hp/isos/respin/rtl8723bs_4.12.0_amd64.deb" -p "/home/hp/isos/respin/linuxium-install-UCM-files.sh" -p "/home/hp/isos/respin/linuxium-install-broadcom-drivers.sh" -p "/home/hp/isos/respin/wrapper-linuxium-install-UCM-files.sh" -p "/home/hp/isos/respin/wrapper-linuxium-install-broadcom-drivers.sh" -b rEFInd' ...
    '/home/hp/Downloads/kodibuntu-14-0rc-64-bit-multi-ubu.iso' must be an Ubuntu (or Ubuntu flavour), Linux Mint, elementary, or neon desktop ISO.
    [/code]

    many thanx in advance was a old kodibuntu iso i use :)

    ReplyDelete
    Replies
    1. Your ISO 'kodibuntu-14-0rc-64-bit-multi-ubu.iso' is not from http://releases.ubuntu.com, https://www.ubuntu.com/download/ubuntu-flavours, https://www.linuxmint.com/download.php, https://neon.kde.org/download or https://elementary.io i.e. it can only be an official desktop 64-bit Ubuntu, Ubuntu flavour, Linux Mint, KDE neon or elementary OS ISO.

      Delete
    2. did same with this one to :


      code]
      Script './isorespin.sh' called with '-i /home/hp/isos/budgie-remix-16.10-amd64.iso -k 4.12.0-041200rc6 -p "/home/hp/isos/respin/rtl8723bs_4.12.0_amd64.deb" -p "/home/hp/isos/respin/linuxium-install-UCM-files.sh" -p "/home/hp/isos/respin/linuxium-install-broadcom-drivers.sh" -p "/home/hp/isos/respin/wrapper-linuxium-install-UCM-files.sh" -p "/home/hp/isos/respin/wrapper-linuxium-install-broadcom-drivers.sh" -l "/home/hp/isos/respin/rtl8723bs_4.12.0_amd64.deb" -b rEFInd' ...
      '/home/hp/isos/budgie-remix-16.10-amd64.iso' must be an Ubuntu (or Ubuntu flavour), Linux Mint, elementary, or neon desktop ISO.

      [/code]

      Delete
    3. Budgie Remix isn't an official Ubuntu flavour. If you want Ubuntu Budgie use the ISO from 'http://cdimages.ubuntu.com/ubuntu-budgie/releases/17.04/release/ubuntu-budgie-17.04-desktop-amd64.iso'.

      Delete
    4. on a side note kodibuntu is a ubuntu flava to :)

      Delete
    5. However 'Kodibuntu' is not an official flavour (see https://www.ubuntu.com/download/ubuntu-flavours).

      Delete
    6. thanx grabbing now :)

      update this error :

      [code]
      Script './isorespin.sh' called with '-i /home/hp/Downloads/ubuntu-budgie-17.04-desktop-amd64.iso -k 4.12.0-041200rc6 -p "/home/hp/isos/respin/rtl8723bs_4.12.0_amd64.deb" -p "/home/hp/isos/respin/linuxium-install-UCM-files.sh" -p "/home/hp/isos/respin/linuxium-install-broadcom-drivers.sh" -p "/home/hp/isos/respin/wrapper-linuxium-install-UCM-files.sh" -p "/home/hp/isos/respin/wrapper-linuxium-install-broadcom-drivers.sh" -l "/home/hp/isos/respin/rtl8723bs_4.12.0_amd64.deb" -b rEFInd' ...
      Work directory 'isorespin' used ...
      Kernel '4.12.0-041200rc6' not found in 'http://kernel.ubuntu.com/~kernel-ppa/mainline'.

      [/code]

      am i better just choosing latest for a win10tab

      Delete
    7. If you look at 'http://kernel.ubuntu.com/~kernel-ppa/mainline/?C=M;O=A' (see the documentation on '-k or --kernel' above) you will see that the kernel '4.12.0-041200rc6' you specified doesn't exist and you probably meant 'v4.12-rc6'. Also you have repeatedly used the '-p' option when you should have used '-l' or -f' depending on the file and you are also missing the '-c' options. So the command should be:

      ./isorespin.sh' -i /home/hp/Downloads/ubuntu-budgie-17.04-desktop-amd64.iso -k v4.12-rc6 -l /home/hp/isos/respin/rtl8723bs_4.12.0_amd64.deb -f /home/hp/isos/respin/linuxium-install-UCM-files.sh -f /home/hp/isos/respin/linuxium-install-broadcom-drivers.sh -f /home/hp/isos/respin/wrapper-linuxium-install-UCM-files.sh -f /home/hp/isos/respin/wrapper-linuxium-install-broadcom-drivers.sh -c wrapper-linuxium-install-UCM-files.sh -c wrapper-linuxium-install-broadcom-drivers.sh -b rEFInd

      Delete
    8. thanx for reply :)

      sorry if im been dumb but where would i edit to copy and paste what your have given i norm use the gui to build iso via .sh in terminal , can i just edit a file in place into it the above you gave with correct -f & -c ?

      Delete
    9. Yes just open up a terminal like before and instead of entering './isorespin.sh' enter the full command './isorespin.sh -i /home/hp/Downloads/ubuntu-budgie-17.04-desktop-amd64.iso -k v4.12-rc6 -l /home/hp/isos/respin/rtl8723bs_4.12.0_amd64.deb -f /home/hp/isos/respin/linuxium-install-UCM-files.sh -f /home/hp/isos/respin/linuxium-install-broadcom-drivers.sh -f /home/hp/isos/respin/wrapper-linuxium-install-UCM-files.sh -f /home/hp/isos/respin/wrapper-linuxium-install-broadcom-drivers.sh -c wrapper-linuxium-install-UCM-files.sh -c wrapper-linuxium-install-broadcom-drivers.sh -b rEFInd' obviously without the "'" quotes and you can use copy and paste to do this.

      Delete
  10. mucho gratis sir think were good now and will report back once i install :)

    ReplyDelete
  11. First of all - thank you for your great work and the efforts you put into it. It is indeed impressive.

    I am trying to install Linux Mint 18.1 on an ODYS Shape Pro whith the help of your scripts. The Kernel 4.12 (released a few days ago, no RC) has the MMC problem again with the RPMB partition on eMMCs, so the machine reacts FAR too sluggish to be useful. Kernel 4.11.8 on the other hand lacks support for the WLAN (RTL8723BS) including bluetooth, also the battery is not recognized, only if it is charging.

    Do you know of a patch for 4.12 to get rid of the encrypted RPMB Partition problem that could help getting the machine into a usable state? SD-Card slot and audio not working can be bypassed with external hardware, WLAN would be nice, but the battery state is important, I think.

    Thanks again for your great work!

    Greetings from Vienna, Austria, EU, Earth.

    ReplyDelete
    Replies
    1. If you want to build your own kernel (which you can do using 'isorespin.sh see http://linuxiumcomau.blogspot.com.au/2017/06/compiling-your-own-kernel-using.html) you can try adding this patch 'https://patchwork.kernel.org/patch/4710641/' although it may need porting to the kernel version you are compiling.

      Delete
    2. Thank you for the info, will try this next weekend.

      Delete
  12. Hi
    Struggling a little getting an Invalid option

    /usr/local/bin/isorespin.sh -i /home/retromint/linuxmint-18.2-mate-64bit.iso -u -l /home/retromint/rtl8723bs_4.12.0_amd64.deb -f /home/retromint/linuxium-install-UCM-files.sh -f /home/retromint/wrapper-linuxium-install-UCM-files.sh-c /home/retromint/wrapper-linuxium-install-UCM-files.sh
    /usr/local/bin/isorespin.sh: Invalid option '-l /home/retromint/rtl8723bs_4.12.0_amd64.deb'.
    Usage: /usr/local/bin/isorespin.sh [ -h | -v ]

    ReplyDelete
    Replies
    1. Did you post the correct command and error message as if I use the command you posted I get the error 'Flag not specified for '/home/retromint/wrapper-linuxium-install-UCM-files.sh''. This is because you are missing the space in front of the '-c'. Can you also make sure you download and use the version of my script from 'https://goo.gl/ZnUd6H' just in case you are using an older version?

      Delete
    2. Sorry been away for a few days
      Yes CRC32 for both scripts are showing as 948C4DD6

      Full error
      retromint@retro-VB ~ $ sudo /usr/local/bin/isorespin.sh -i /home/retromint/linuxmint-18.2-mate-64bit.iso -u -l /home/retromint/rtl8723bs_4.12.0_amd64.deb -f /home/retromint/linuxium-install-UCM-files.sh -f /home/retromint/wrapper-linuxium-install-UCM-files.sh -c /home/retromint/wrapper-linuxium-install-UCM-files.sh
      /usr/local/bin/isorespin.sh: Invalid option '-l /home/retromint/rtl8723bs_4.12.0_amd64.deb'.
      Usage: /usr/local/bin/isorespin.sh [ -h | -v ]
      /usr/local/bin/isorespin.sh -i [ [ -u | -k ] | -r " ... " | -p " ... " | -l " ... " | -f " | ... " | [ -s MB | GB ] | [ -b GRUB | rEFInd ] ...
      /usr/local/bin/isorespin.sh ... -w | -d " ... " | -e " ... " | -c " ... " | -o " | ... " | -g "" | " ... " ]

      am running mint in a VM
      I think I followed all the steps correctly ( I do not use linux a lot )

      Delete
    3. Sorry I copy and pasted command into libraoffice, it looks like I also copied some formatting (shown as grey blocks) once I deleted these it ran with errors:-
      created 168180 files
      created 21261 directories
      created 48197 symlinks
      created 81 devices
      created 0 fifos
      Extracting isorespin files ...
      Updating bootloader/bootmanager ...
      chroot: failed to run command 'apt-cache': Exec format error
      chroot: failed to run command 'apt-cache': Exec format error
      mv: cannot stat 'iso-chroot/usr/src/grub-efi-ia32-bin*.deb': No such file or directory
      mv: cannot stat 'iso-chroot/usr/src/grub-efi-ia32*.deb': No such file or directory
      chroot: failed to run command 'apt-cache': Exec format error
      chroot: failed to run command 'apt-cache': Exec format error
      Fetching mainline kernel packages ...
      Installing mainline kernel packages ...
      /usr/local/bin/isorespin.sh: Cannot find mainline kernel.

      Delete
    4. The 'crc32' for 'isorespin.sh' is '948c40d6' not '948C4DD6' ... was it a typo?

      Also are you running natively on Linux and if so what distro or are you running in a VM?

      Delete
    5. Thank you for reply
      Host is Win7 - 64bit (i7 16GB ram)
      Guest is Mint 18.1 - xfce 32-bit
      Sorry typo D should be 0 (Zero)

      Delete
    6. I've not tried running the script in a Linux Mint VM. It works with an Ubuntu VM using Oracle's VirtualBox on Windows so can you create a Linux VM from a standard Ubuntu 64-bit desktop ISO and install to a VDI of fixed size (minimum 20GB) then start the new Ubuntu VM and install 'Oracle VM VirtualBox Extension Pack' before installing the two required packages of 'squashfs-tools' and 'xorriso' and then running my 'isorespin.sh' script?

      Delete
    7. Thank you I think that worked ( looks good )
      unsure what my Z3735G 1GB/16GB will be like but it has to better than Win10 that cannot update due to one USB for charge and data (needs a USB drive and power to update and powered OTG do not work)

      ISO image produced: 921176 sectors
      Written to medium : 921176 sectors at LBA 0
      Writing to 'stdio:../../linuxium-v4.13-rc1-linuxmint-18.2-mate-64bit.iso' completed successfully.

      /home/vm/isorespin.sh: Respun ISO created as 'linuxium-v4.13-rc1-linuxmint-18.2-mate-64bit.iso' ... see logfile 'isorespin.log' for details.
      vm@vm-VirtualBox:~$

      Delete
    8. You will probably find that Lubuntu would be a better option for your device if it only has 1GB RAM.

      Delete
    9. Thank you again for you work and help

      Delete
    10. Tried Lubuntu but fails to install grub (does not ask for wifi during install)
      Mint did work but was not great on a touchscreen (will need to try harder with it)

      Delete
    11. I get no sound or bluetooth I also see errors
      Command run ...
      # wrapper-linuxium-install-broadcom-drivers.sh
      bash: /usr/local/bin/wrapper-linuxium-install-broadcom-drivers.sh: Permission denied

      Have tried with file in /usr/local/bin and home folder same error

      Delete
    12. Lubuntu requires internet to successfully install (see notes in my documentation). For the 'Permission denied' error you need to set execute first on the script before respinning (enter 'sudo chmod 755 wrapper-linuxium-install-broadcom-drivers.sh').

      Delete
  13. Hello there!

    I am trying to set up a Beelink Z83 with a Cherry Tail Atom X8350 processor. WiFi and Sound via HDMI is cruicial for my application, sound via analogue jack would be a lovely addon.

    I have downloaded from one of your recent blog posts this file: linuxium-v4.12-rc7-ubuntu-17.04-desktop-amd64.iso
    md5: f1c5d906d6a335f2c51455d713048ad8

    Ths 17.04 works like a charm. However, for my business case, i would like to use an ubuntu LTS version. So i downloaded 16.04 desktop, downloaded the 4 additional files and 2 wrapper scripts to the same folder with your iso tool, and ran the following command:

    bash isorespin.sh -i ubuntu-16.04.1-desktop-amd64.iso -u -l rtl8723bs_4.12.0_amd64.deb -f linuxium-install-UCM-files.sh -f wrapper-linuxium-install-UCM-files.sh -c wrapper-linuxium-install-UCM-files.sh

    This ran through fine and i copied the resulting linuxium-v4.12-ubuntu-16.04.1-desktop-amd64.iso to my usb stick and booted from it.

    However, the onboard wifi is not found now. I hoped I would receive a ubuntu 16.04 with same kernel and patches that you already put into your 17.04 image - but i failed.

    Am i doing something wrong, or is it not possible to make a 16.04 work with a 4.12 kernel and wifi working?

    Thanks for your time and efforts - highly appreciated.

    Regards,
    Lucas

    ReplyDelete
    Replies
    1. I found out that when i run the linuxium-install* scripts manually in the live system, wifi starts to work immediately.

      Now i tried to install the linuxium i made on the device. However, it just randomly freezes totally (clock stops, everything stops). Any hints what I may still be missing?

      Delete
    2. Try booting using the kernel boot parameter of 'intel_idle.max_cstate=1' as this may help with kernel freezes.

      Delete
  14. Hi, first, thanks much for this script...
    (tried "dmesg | pastebinit", but apparently it doesn't run behind a proxy).

    I'm working with an Azulle Access Plus (AAP) which uses a Cherry Trail. I bought the unit and overwrote W-10 with the .iso that AzulleTech provided. That unit runs my application fine (its Ubuntu16.04.2 LTS with a 4.5.7-archermind-standard+ kernel).
    Then I bought two more, and was not able to get the iso, as their tech support pointed me to the standard Ubuntu16.04.2LTS image.

    Using the off-the-shelf Ubuntu16.04.2LTS, the system boots, and as long as I don't run my application its happy. When I start my application (networked audio/video encode/decode using gstreamer) it runs for about 15 minutes just fine (good bi-directional audio and video), then the entire system freezes (recoverable only by power-cycle).

    Then I read about the Cherry Trail issues with Ubuntu and fortunately found your script. I run your script with
    -i isofile -k v4.11 -g "intel_idle.max_cstate=1" successfully, but after installing it on the AAP, it still freezes. Note that this application has been running on other linux systems for quite some time. Also note that the system does not freeze if my application is not run.
    Any thoughts? I am trying to get Azulle to give me the original .iso, but I'd prefer to be up-to-date if possible. I can try 4.12, but I had thought 4.11 pulled all Cherry Trail stuff into mainline.
    Thanks in advance,
    Ed

    ReplyDelete
    Replies
    1. A bit more information, which rules out the hardware as a potential issue, and IMHO points clearly to the distribution or kernel...

      I did a raw copy of the internal disk of the good system and transferred that image to the system that was freezing.

      Seems to me, since both are running 16.04.2 LTS, that points to the kernel; but since I don't know where the kernel came from (it was part of the .iso I got from Azulle) I can't isolate anything further.

      Any thoughts?

      Delete
    2. One small bit of "key" information I left out in the above reply was that after installing the new image on the system that was freezing, it froze no more.

      Delete
  15. I was able to post dmesg output to pastebin.com...
    https://pastebin.com/yfRpbFBU

    ReplyDelete
    Replies
    1. The v4.12 kernel has additional Intel Atom patches and looking at your 'dmesg' you should probably add my 'linuxium-install-UCM-files.sh' (see above) to fix the 'ASoC: no backend DAIs' messages. Unfortunately freezes are not solved although the 'cstate' kernel boot parameter helps. Perhaps it is freezing due to thermal throttling? Take a look at http://linuxiumcomau.blogspot.com.au/2017/03/ubuntu-16042-and-ubuntu-1704-beta-1.html?showComment=1489005628439#c7389843581916420878 and see if this help.

      Delete
    2. Ok, I'll give all this a try and report back.
      BTW...I was adding a bit more info (see above) as you were apparently replying. Thanks much.

      Delete
    3. Here's the dmesg output of the system that works (now I have two of them)... https://pastebin.com/ZT6Ep11v

      Delete
    4. BTW, I had tried 4.12 last night and that made no difference either.

      Delete
    5. Well, still no luck.. Same symptom... Freeze after about 5 minutes of running.

      isorespin.log: https://pastebin.com/wkj0JJka
      dmesg output: https://pastebin.com/Ryi6mQfY

      I'd sure appreciate any thoughts or suggestions.

      Delete
    6. After posting the previous results I noticed that the "-g" option of the isorespin.sh doesn't appear to have inserted the kernel command line parameters. I manually added them through /etc/default/grub, but still no difference.

      Delete
    7. I ran 'isorespin.sh' with the '-g' option you used (as seen in the log you posted) and it did insert the kernel boot parameters correctly. However your 'dmesg' shows you running a kernel without them. Can you check that you are booting the correct kernel as another indication is that the ALSA UCM files are missing and you added them according to the log (by running 'wrapper-linuxium-install-UCM-files.sh')?

      Delete
    8. Ok, I am behind a proxy, so that was causing some problems. Seems like the different "sudo wget" and "sudo chroot iso_chroot /bin/bashrc" commands don't do well without some help there. I set up everything I could think of to get over this hump...
      1. Set up /etc/apt/apt.conf to be proxy aware.
      2. Set http_proxy and https_proxy in /etc/profile.
      3. Set up .wgetrc in $HOME of invoking user.
      4. Modify isorespin.sh (lemme know if you want a copy):
      a. Add -E to "sudo" for wget.
      b. Add -e use_proxy=yes -e http_proxy=PROXY to wget lines.

      After all that, the UCM files properly installed, but still no kernel command line additions. So, again I added them manually, but my unit *still* hangs after 5-10 minutes of multimedia.

      dmesg: https://pastebin.com/cMwR4gU5
      isorespin.log: https://pastebin.com/5qxr72DT


      It doesn't seem to be a heat problem either; the unit isn't very warm. Actually, the one unit I have that works with the older "source-unknown" kernel shows the cores at about 74C, where this kernel hangs with cores at around 67C.

      Delete
    9. The 'dmesg' shows the kernel boot parameters so do you mean after installation the kernel boot parameters that were there for LiveUSB boot are not present in the installed system?

      Delete
    10. Right... if I boot from USB and just "try" ubuntu, the boot parameters from the isorespin.sh command line are there; but when I install the system (I just erase and write over everything that was previously on disk) the boot parameters specified on the isorespin.sh command line are not there. Here are the two dmesg outputs:
      dmesg.usbboot: https://pastebin.com/TPra5MCY
      dmesg.install: https://pastebin.com/CzuxJTWr

      Again... thanks for your help.

      Delete
    11. By the way, I'm able to reproduce the freeze by going to webglsamples.org/aquarium/aquarium.html and setting the "Number of Fish" to 500 or more (then just wait 10-15 minutes).

      Delete
    12. When the ISO installs Ubuntu the GRUB installer creates a new '/etc/default/grub' file rather than copying the existing one which is why the kernel boot parameters are present on the LiveUSB and not when installed. I'm in two minds as to whether it is 'correct' to reflect the '-g' option kernel boot parameters once installed and on how to do it if determined 'appropriate'. Do you have any thoughts or comments?

      Regarding the freeze issue perhaps you can post your feedback on Kernel.org Bugzilla against the existing bug report 'https://bugzilla.kernel.org/show_bug.cgi?id=109051'?

      Delete
    13. Thanks, I'll post some feedback.
      Regarding GRUB options...
      My bootloader experience is on non-Intel systems, so keeping that in mind...
      If the install actually preserved the current boot parameters then I'd say maybe you should leave it like that (so whatever was there is not lost); however, if the new grub just installs a standard /etc/default/grub file anyway, then it seems to be appropriate for the -g options of your script to be put in with the install.
      Of course, that assumes there is some way of doing that!...
      Maybe a script (-c) could copy the initial /etc/default/grub to grub.bak, and move a new one (with -g options) to that location.

      Delete
    14. Ok, some new data (with a question)...
      Based on some comments I saw at https://bugzilla.kernel.org/show_bug.cgi?id=109051, I went back to 4.12 (rather than 4.12.2), and instead of installing to internal disk, I just ran from USB. I ran the aquarium stuff for about 5 hours without a glitch. Then I took that image and installed to internal disk, (forced the same GRUB options) and ran the aquarium and it died in about 15 minutes.

      Would you expect there to be any difference in runtime between the two scenarios?

      Delete
    15. I've fixed the GRUB issue and it will be included in the next release of my script.

      For the random freeze it has been reported as random so 5 hours without freeze and 15 minutes with freeze isn't necessarily unusual in the same environment. What would be interesting was if the timing (and especially lack of freeze) can be uniformly repeated on the two scenarios as this would be a pointer of where to look next.

      Delete
  16. Is it possible to get this working for a server iso instead of a desktop iso? I'd love to get Ubuntu Server 17 working. Thanks for your answer, even if the answer is no.

    ReplyDelete
    Replies
    1. No. But as a workaround you can first respin and install the desktop ISO and then purge the desktop packages and install the required server packages (see http://www.linuxium.com.au/how-tos/creatinganubuntuserveronaz3735fdevice for an example). You might even be able to do this through respinning!

      Delete
  17. Hi, I tried using your command to install UCM files using the -f and -c cmdline args, and the scripts are failing to run - isorespin.log is showing lots of 'sudo: unable to resolve host ' errors. Do you know what could cause this?

    ReplyDelete
    Replies
    1. Are you connected to the internet before running the scripts?

      Delete
    2. Yes, I'm connected to internet and can ping google.com for example. The hostname referenced in 'sudo: unable to resolve host ' is the local hostname of the system where I'm running isorespin, which is a vanilla install of Ubuntu 16.04 64-bit - it's directly installed (not a VM), if that makes a difference.

      Delete
    3. Running sudo outside isorespin works fine, but any time sudo is run in a script invoked by -c it shows the 'unable to resolve host [local hostname]' error in the log.

      Delete
    4. Are you running the 'linuxium-install-UCM-files.sh' using the '-c' option as if so you should be running the 'wrapper-linuxium-install-UCM-files.sh' as that fixes the hostname issue. You still have to load the 'linuxium-install-UCM-files.sh' script so the respin command would be along the lines of 'isorespin.sh -i ...your various options... -f linuxium-install-UCM-files.sh -f wrapper-linuxium-install-UCM-files.sh -c wrapper-linuxium-install-UCM-files.sh'.

      Delete
    5. concrete_d, if you're behind a proxy, see my reply above...

      Delete
  18. Thank you for your effort. Unfortunately anything I have tried ended up at different progress advancement with the message like this:

    write_file: failed to create file squashfs-root/var/lib/dpkg/info/libfdisk1:amd64.triggers, because Invalid argument

    Executed on Ubuntu 16.04.2 LTS (amd64)

    What could be the reason?

    ReplyDelete
    Replies
    1. It is not an error I'm familiar with. Can you provide more details like: are you running natively or on a VM (and in which case what); what version and what is the md5sum of my 'isorespin.sh' script; what ISO are you trying to respin; what are the contents of 'isorespin.log' after encountering the error; what is the full error message?

      Delete
    2. I executed your script on the NTFS partition with fixed attributes/ownership. Same machine, ext4 working dir and no problems.
      Interestingly on another machine, also NTFS work dir, it executes without any such messages. Probably a bit different set of attributes.

      Delete
    3. Interesting. I'd only use EXT4 for the working directory though.

      Delete