Friday 2 June 2017

New functionality and simplification of 'isorespin.sh'

UpdateThis work is superseded by my 'isorespin.sh' script which can respin an official ISO suitable for use on Intel Atom devices.


I've updated my 'isorespin.sh' script to include new functionality and simplified its invocation. Besides uploading files to the ISO I've included the ability to upload directories and conversely I've also added the ability to output both files and directories. The other key functionality I've added is the ability to run commands as part of building the ISO and the significance of this is explained in an example below.


The GUI menu is now simplified through adding a sub-menu for the more advanced options that may not be so frequently used. Also adding persistence no longer requires specifying a boot manager. Finally I recommend installing the script by copying it to '/usr/local/bin' as this makes it readily accessible.

Although more detailed documentation is covered by earlier posts as a reminder you can respin all recognized desktop 64-bit Ubuntu and Ubuntu flavoured ISOs (https://www.ubuntu.com/download/ubuntu-flavours), Linux Mint and LDME ISOs (https://www.linuxmint.com/download.php), Debian 'Live install' ISOs (https://www.debian.org/CD/live), KDE neon ISOs (https://neon.kde.org/download), elementary OS ISOs (https://elementary.io) and Kali ISOs (https://www.kali.org/downloads) and in doing so will automatically add a 32-bit bootloader as well as the 64-bit bootloader making it possible to boot the respun ISO on Intel Atom devices.

The basic requirements are unchanged in that you will need to use a Linux machine (or a Linux virtual machine on Windows) with packages like 'squashfs-tools' and 'xorriso' installed (the script will check what is required and notify you if anything is missing) together with 'zenity' if you want the GUI. You will also need at least 10GB of free space and a working internet connection if you want to update the kernel. And on that note the '-u' or '--update' flag will update an Ubuntu or Linux Mint ISO with the latest mainline kernel but when used with a Debian or LMDE ISO it will update the kernel with the Canonical DKMS patches for HDMI audio and RTL8723BS wifi. As a result the '-k' or '--kernel' option is only allowed for Ubuntu and Linux Mint ISOs and any of the Ubuntu Kernel Team kernel builds found at http://kernel.ubuntu.com/~kernel-ppa/mainline can be specified by using the directory/folder name without the trailing '/' (as in '--kernel v4.12-rc3'). Also if selecting the rEFInd bootmanager (http://www.rodsbooks.com/refind) and it is in the directory from where the script is run it won't be downloaded again from https://sourceforge.net/projects/refind/files but the local copy used instead regardless of its version. And once the ISO has been respun it can be written to a USB using the standard 'dd' command.

For anyone brave enough to try an installation of the respun ISO you may encounter the usual issues with installing a bootloader. First try installing with a working internet connection as this resolves many of the problems. However if is still doesn't work I've written an additional (updated) script 'linuxium-install-bootloader.sh' that you can try running or use the commands within the script as a guide to allow you to install manually. Note that running the script still requires an internet connection and the target Linux partition needs to be already mounted on '/target' with the target boot partition mounted on '/target/boot/efi'.

Currently even after respinning your ISO you may find that some wifi/bluetooth driver and userspace files are still required to get them working. Previously I've provided additional scripts and packages to support RTL8723BS and various Broadcom chipsets as well as the ALSA UCM files for some audio chipsets used by headphones. And whilst they can be included as files when respinning the ISO they still need to be run on booting the ISO as a 'LiveCD' to activate the additional functionality. This new version of the respin script includes the 'command' option to address this which and it takes its argument and runs it as a 'bash' command.


Simple commands are straight forward. However installing the UCM files by just uploading and running the script is slightly more complex. This is because the script requires the internet and therefore a valid host name and the script must be run in the same directory it resides in. Also any quotes used within command arguments need delimiting.


In more complex cases it is simpler to write a shell script to execute the commands required. An additional command can always be run to remove the 'wrapper' script if desired. See my separate post for an additional example of the power of this new functionality.

The new version of the script can be downloaded from 'isorespin.sh' and any issues or improvement suggestions are welcome.

If you find my work useful then please donate using the following link http://goo.gl/nXWSGf as everything helps with development costs.

26 comments:

Linuxer-hh said...

Issue detected? --> I tried the new version with Lubuntu and LinuxMint-Mate (1st with, 2nd with and without kernel update). Boot-Menue appears but booting comes to a complete stop shortly afterwards. Device is a bay-trail no-name tablet on which distributions done with earlier isorespin-versions did boot.

Linuxium said...

V4.12-rc3 contains a regression bug which prevents booting on BYT and CHT devices so use '-k v4.12-rc2' or wait until Sunday/Monday and try rc4.

Linuxer-hh said...

Ups, I guess I should have done some more reading first... Thanks for your reply; it boots now. Let's see if I can get it installed as well.

Linuxer-hh said...

It's me again...
Sorry, maybe I lack some basic skills here. The system installed well until I reached the point with grub. After crash of the installer I tried your script, but it does also not run through. Any advice would be great!

..sudo mount /dev/mmcblk1p2 /target/
..sudo mount /dev/mmcblk1p1 /target/boot/efi/

mint@mint ~/Downloads $ sudo ./linuxium-install-bootloader.sh
Ign:1 cdrom://Linux Mint 18.1 _Serena_ - Release amd64 20161213 xenial InRelease
Err:2 cdrom://Linux Mint 18.1 _Serena_ - Release amd64 20161213 xenial Release
Please use apt-cdrom to make this CD-ROM recognized by APT. apt-get update cannot be used to add new CD-ROMs
Hit:3 http://archive.ubuntu.com/ubuntu xenial InRelease
Hit:4 http://archive.canonical.com/ubuntu xenial InRelease
Get:5 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:6 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Ign:7 http://packages.linuxmint.com serena InRelease
Hit:8 http://packages.linuxmint.com serena Release
Get:10 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Reading package lists... Done
E: The repository 'cdrom://Linux Mint 18.1 _Serena_ - Release amd64 20161213 xenial Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
./linuxium-install-bootloader.sh: 42: [: 32: unexpected operator
grub-install.real: error: /usr/lib/grub/x86_64-efi/modinfo.sh doesn't exist. Please specify --target or --directory.
/usr/sbin/grub-mkconfig: 252: /usr/sbin/grub-mkconfig: cannot create /boot/grub/grub.cfg.new: Directory nonexistent
mint@mint ~/Downloads $

Linuxium said...

First did you try performing the installation whilst connected to the internet as this fixes most situations?

Given the installation has finished and assuming you have done nothing since, try booting again from the LiveUSB and enter these command:

sudo mount /dev/mmcblk1p2 /target/
sudo mount /dev/mmcblk1p1 /target/boot/efi/
sudo chroot /target mount -t proc proc /proc
sudo chroot /target mount -t sysfs sysfs /sys
sudo mount --bind /dev /target/dev
sudo mount --bind /run /target/run
sudo chroot /target apt update
sudo chroot /target apt -y purge grub-efi-amd64 grub-efi-amd64-bin grub-efi-amd64-signed
sudo chroot /target apt -y purge shim shim-signed
sudo chroot /target apt -y purge grub-efi-ia32 grub-efi-ia32-bin
sudo chroot /target apt -y purge grub-common grub2-common
sudo chroot /target rm -rf /boot/grub/
sudo chroot /target rm -rf /boot/efi/EFI/ubuntu/
sudo chroot /target apt -y install grub-common grub2-common
sudo chroot /target apt -y install grub2-common grub-efi-ia32 grub-efi-ia32-bin
sudo chroot /target grub-install --target=i386-efi ${BOOTLOAD_PARTITION} --efi-directory=/boot/efi/ --boot-directory=/boot/
sudo chroot /target grub-mkconfig -o /boot/grub/grub.cfg
sudo chroot /target umount /sys
sudo chroot /target umount /proc
sudo umount /target/run
sudo umount /target/dev


You will need a working internet connection for this to work. If there are no errors, reboot and see what happens (as you may then need to fix up NVRAM). Otherwise please post any error encountered.

Linuxer-hh said...

Hi there,

Worked Fine!
- I did had an internet connection on the 1st try
- On your script, only where you proposed an "apt", I needed to switch to "apt-get", as apt returned only an info about how to use that command...

--> Thank you very much!!!

Linuxium said...

Thanks. I'll update the 'linuxium-install-bootloader.sh' script with this feedback and post for the benefit of others.

Linuxium said...

Please provide full details about the error so I can try and reproduce and investigate.

Anonymous said...

I think i've found a bug in the gui and also I'm struggling to produce a valid (recognisable) iso

Can't fit in 4096 characters
https://pastebin.com/qJ3DmP1b

Linuxium said...

Thanks. Yes you have found a bug and your error report is excellent and has really helped in fixing the issue. I've posted an updated version (v6.03.0). I've also added additional checking that should help with the 'cp' issue you encountered which I think happens when a download fails (for whatever reason) and is typically unreproduceable as it disappears when the script is re-run.

Anonymous said...

Ok thanks. I have it working now.
BTW your script can run on Arch Linux if I modify the PATH.
PATH=/usr/sbin:/sbin:/bin:$PATH ./isorespin.sh
Without that the many chroots called in the script weren't working as Arch's path is missing a lot of Ubuntu's entries.

It would be nice if you could put this all on a git repo or similar. ATM the instructions (some conflicting) are spread around your blog pages).

But anyway good work and I've sent some beer tokens

Linuxium said...

Thanks. I'll write an updated guide with the latest links. Actually my blog is intended to be the 'dynamic' update content partner to my 'static' core content pages on my website Typically I publish my development on my blog then once released and stable I published on my website. This approach is meant to address any conflicting information and centralize it in once place.

Kamal said...

Hi Linuxium, I have the Lenovo Ideapad 100S 11IBY (Intel Atom). I installed Lubuntu using a bootable USB that I made using your patched kernel iso and it ran perfectly. However, I installed xfce4 desktop environment from within terminal after which the laptop failed to boot. On the boot options menu I get two options (1) "Ubuntu" and (2) "Ubuntu", yes they are both same words. The laptop fails to boot using any of these two options. I tried to boot using more linuxium bootable usbs but they all failed to boot. The laptop is unusable now. Can you please help? Some people suggested that because I updated the lubuntu, the grub might have been replaced new grub which is causing the problem. Any help will be appreciated.

Linuxium said...

Just use 'efibootmgr' from a LiveUSB to remove both of them and then use it to recreate a single entry.

Kamal said...

My laptop Lenovo Ideapad 100S 11iby has ceased to boot from even UEFI bootable USBs. This happened after I installed an extra desktop environment on existing linuxium installed Lubuntu. Do you have any idea of how to make a USB boot in such a situation. Perhaps some BIOS setting or some way to access the storage without booting?

Linuxium said...

Assuming your laptop's eMMC (storage as in your disk drive) has corrupted try downloading 'gparted-live-0.28.1-1-amd64.iso' from 'http://gparted.org/download.php' and write to a USB with 'dd'. Then see if you can boot from it and if so use 'gparted' to wipe your eMMC by creating a new partition table 'Device->Create Partition Table->msdos->Apply'. You should then be able to boot from an Ubuntu ISO USB.

Kamal said...

I made a gparted live USB with 'dd' and went to 'EFI USB' in the boot menu but it just says 'boot device missing'. I tried with both secure boot 'on' and 'off'. I also tried with:
(1) Setup Mode,
(2) Restore all factory keys,
(3) Load default settings.
Neither of these combinations work. I am wondering how is this even possible. Is there some particular UEFI setting I need to setup? Or is there a way a access the storage using a USB cable or something like that. I know that eMMC is soldered onto motherboard.

Kamal said...

One more thing, I cloned my entire eMMC onto an HDD before installling ubuntu on my laptop. Do you think it is possible to boot from the HDD via a SATA-to-USB cable? (It is proper clone made using CloneZilla.)

Linuxium said...

Try this 'https://forums.lenovo.com/t5/Lenovo-IdeaPad-100-305-500-Edge/Ideapad-100S-bricked-by-Windows-update-Solved/m-p/3377245#M38858'

nhk said...

so Ubuntu respin works via USB no worries, it seemed to install OK... but on reboot obviously its failing with GRUB on my Apollo Lake Ezybook 3 Pro. I need ReFIND... the section has been removed from the tutorial? I tried efibootmgr and removed the previous windows entry, but it still fails with nothing but a black screen and white cursor in top left. Any tips on how to get refind installed on there so I can actually boot into the OS without a usb stick connected? none of the information seems to be helping anywhere I've looked so far

Linuxium said...

The 'rEFInd' link above includes documentation on installation (see 'http://www.rodsbooks.com/refind/installing.html') so I didn't add anything else to the tutorial. Do you think there is a need for additional instructions?

Kamal said...

Hi, that link has an eloborate BIOS but my laptop does not (Ideapad 100S 11IBY). I tried making GPT only bootable USB using your respun iso "linuxium-atom-v4.13.0-9-ubuntu-17.10-10SEP17-desktop-amd64" but the laptop refuses to identify the EFI boot USB. It keeps going into a loop "ubuntu boot faile" followed by,
"Default boot deviceMissing or boot failed. Insert recovery media and hit any key, Then select Boot Manager to choose a new Boot Device or to Boot Recovery Media."

Kamal said...

Please, I will try anything at this point. The laptop is unusable.

Linuxium said...

I suggest reflashing the BIOS as per the manufacturer's recommended method.

Kamal said...

Is it possible without booting an OS, I mean directly from the bios?

Linuxium said...

It will depend on your device manufacturer but it is typically possible. Otherwise you can flash the BIOS chip with an EEPROM programmer.

Post a Comment