Sunday, 6 August 2017

Respinning security distros and upgrading packages

A while back I dropped Debian and Debian based ISO support from my 'isorespin.sh' script as the release of Debian 9 Stretch uses a v4.9 kernel rather than a v3.16 kernel meaning that the kernel cannot be upgraded with Canonical's HDMI and RTL8723BS DKMS support. I also dropped their support because I do not agree with using a kernel compiled for one distro's userland with a different distro's userland as for example in using an Ubuntu kernel to boot a Fedora ISO.

This meant no more respinning Kali ISOs but since I'm again receiving requests for its support it got me thinking about what Ubuntu-based security distros existed. So following some research I've added support for BackBox Linux a 'penetration testing and security assessment oriented Linux distribution providing a network and systems analysis toolkit'.


Respinning is simple using my latest version of 'isorespin.sh':

Script '/usr/local/bin/isorespin.sh' called with '-i backbox-5-amd64.iso --atom --update' ...
Work directory 'isorespin' used ...
ISO '/home/linuxium/backbox-5-amd64.iso' respun ...
Bootloader 'GRUB' added ...
Kernel updated with mainline kernel version '4.13.0-041300rc3-generic' ...
Local package '/home/linuxium/isorespin/rtl8723bs_4.12.0_amd64.deb' added ...
File '/home/linuxium/isorespin/linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/linuxium-install-broadcom-drivers.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-broadcom-drivers.sh' added ...
Command run ...
# wrapper-linuxium-install-UCM-files.sh
./linuxium-install-UCM-files.sh: Extracting UCM files ...
./linuxium-install-UCM-files.sh: Installing UCM files ...
./linuxium-install-UCM-files.sh: Reloading UCM driver ...
./linuxium-install-UCM-files.sh: Installation of UCM finished 
# wrapper-linuxium-install-broadcom-drivers.sh
./linuxium-install-broadcom-drivers.sh: Extracting Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Reloading Broadcom driver ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom bluetooth service ...
Created symlink /etc/systemd/system/multi-user.target.wants/brcmbt.service, pointing to /lib/systemd/system/brcmbt.service.
./linuxium-install-broadcom-drivers.sh: Starting Broadcom bluetooth service ...
Running in chroot, ignoring request.
./linuxium-install-broadcom-drivers.sh: Installation of Broadcom finished 
Respun ISO created as 'linuxium-v4.13-rc3-backbox-5-amd64.iso'.

I've also had another look at Kali as whilst their official ISOs use a Debian kernel they also offer Kali Metapackages which 'give you the flexibility to install specific subsets of tools based on your particular needs'. Following the documented instructions I looked at how I could update my script to allow the addition of these metapackages when respinning. As a result I've added a new option '--key' to add GPG keys to the APT keyring allowing packages to be downloaded from signed repositories. It is now possible to respin an Ubuntu ISO adding the packages:


There are some restrictions/limitations. Unity isn't supported and I've found adding a GPG key to a 17.04 or 17.10 release fails. Additionally adding the 'kali-linux-full' package results in dependency issues. However it is possible to respin the recently released Ubuntu GNOME 16.04.3 and add 'kali-linux' and 'kali-linux-top10':

Script '/usr/local/bin/isorespin.sh' called with '-i ubuntu-gnome-16.04.3-desktop-amd64.iso --atom -u --key adv --keyserver keyserver.ubuntu.com --recv-keys ED444FF07D8D0BF6 --repository deb http://http.kali.org/kali kali-rolling main contrib non-free -p kali-linux -p kali-linux-top10' ...
Work directory 'isorespin' used ...
ISO '/home/linuxium/ubuntu-gnome-16.04.3-desktop-amd64.iso' respun ...
Bootloader 'GRUB' added ...
Kernel updated with mainline kernel version '4.13.0-041300rc3-generic' ...
Key 'adv --keyserver keyserver.ubuntu.com --recv-keys ED444FF07D8D0BF6' added ...
Repository 'deb http://http.kali.org/kali kali-rolling main contrib non-free' added ...
Package 'kali-linux' added ...
Package 'kali-linux-top10' added ...
Local package '/home/linuxium/isorespin/rtl8723bs_4.12.0_amd64.deb' added ...
File '/home/linuxium/isorespin/linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/linuxium-install-broadcom-drivers.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-broadcom-drivers.sh' added ...
Command run ...
# wrapper-linuxium-install-UCM-files.sh
./linuxium-install-UCM-files.sh: Extracting UCM files ...
./linuxium-install-UCM-files.sh: Installing UCM files ...
./linuxium-install-UCM-files.sh: Reloading UCM driver ...
./linuxium-install-UCM-files.sh: Installation of UCM finished 
# wrapper-linuxium-install-broadcom-drivers.sh
./linuxium-install-broadcom-drivers.sh: Extracting Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Reloading Broadcom driver ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom bluetooth service ...
Created symlink /etc/systemd/system/multi-user.target.wants/brcmbt.service, pointing to /lib/systemd/system/brcmbt.service.
./linuxium-install-broadcom-drivers.sh: Starting Broadcom bluetooth service ...
Running in chroot, ignoring request.
./linuxium-install-broadcom-drivers.sh: Installation of Broadcom finished 
Respun ISO created as 'linuxium-v4.13-rc3-ubuntu-gnome-16.04.3-desktop-amd64.iso'.

Lubuntu is also supported:


and by adding 'kali-desktop-lxde' additional LXDE packages are included (note 'Other'):


Script '/usr/local/bin/isorespin.sh' called with '-i lubuntu-16.04.3-desktop-amd64.iso --atom -u --key adv --keyserver keyserver.ubuntu.com --recv-keys ED444FF07D8D0BF6 --repository deb http://http.kali.org/kali kali-rolling main contrib non-free -p kali-linux -p kali-desktop-lxde -p kali-linux-top10' ...
Work directory 'isorespin' used ...
ISO '/home/linuxium/lubuntu-16.04.3-desktop-amd64.iso' respun ...
Bootloader 'GRUB' added ...
Kernel updated with mainline kernel version '4.13.0-041300rc3-generic' ...
Key 'adv --keyserver keyserver.ubuntu.com --recv-keys ED444FF07D8D0BF6' added ...
Repository 'deb http://http.kali.org/kali kali-rolling main contrib non-free' added ...
Package 'kali-linux' added ...
Package 'kali-desktop-lxde' added ...
Package 'kali-linux-top10' added ...
Local package '/home/linuxium/isorespin/rtl8723bs_4.12.0_amd64.deb' added ...
File '/home/linuxium/isorespin/linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/linuxium-install-broadcom-drivers.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-broadcom-drivers.sh' added ...
Command run ...
# wrapper-linuxium-install-UCM-files.sh
./linuxium-install-UCM-files.sh: Extracting UCM files ...
./linuxium-install-UCM-files.sh: Installing UCM files ...
./linuxium-install-UCM-files.sh: Reloading UCM driver ...
./linuxium-install-UCM-files.sh: Installation of UCM finished 
# wrapper-linuxium-install-broadcom-drivers.sh
./linuxium-install-broadcom-drivers.sh: Extracting Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Reloading Broadcom driver ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom bluetooth service ...
Created symlink /etc/systemd/system/multi-user.target.wants/brcmbt.service, pointing to /lib/systemd/system/brcmbt.service.
./linuxium-install-broadcom-drivers.sh: Starting Broadcom bluetooth service ...
Running in chroot, ignoring request.
./linuxium-install-broadcom-drivers.sh: Installation of Broadcom finished 
Respun ISO created as 'linuxium-v4.13-rc3-lubuntu-16.04.3-desktop-amd64.iso'.

Finally I've added another option '--upgrade' which simply performs an 'apt-get upgrade' on the ISO's packages. So for example having downloaded an Artful daily ISO, I can respin it with the latest packages:


Script '/usr/local/bin/isorespin.sh' called with '-i 030817-artful-desktop-amd64.iso --upgrade --rolling-unstable --atom -s 200MB' ...
Work directory 'isorespin' used ...
ISO '/home/linuxium/030817-artful-desktop-amd64.iso' respun ...
Kernel boot parameters 'persistent' added ...
Bootmanager 'rEFInd' added ...
Distro upgraded ...
Package 'linux-headers-4.12.0-9 linux-headers-4.12.0-9-generic linux-image-4.12.0-9-generic linux-image-extra-4.12.0-9-generic' added ...
Local package '/home/linuxium/isorespin/rtl8723bt_4.12.0_amd64.deb' added ...
File '/home/linuxium/isorespin/linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/linuxium-install-broadcom-drivers.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-broadcom-drivers.sh' added ...
Command run ...
# wrapper-linuxium-install-UCM-files.sh
./linuxium-install-UCM-files.sh: Extracting UCM files ...
./linuxium-install-UCM-files.sh: Installing UCM files ...
./linuxium-install-UCM-files.sh: Reloading UCM driver ...
./linuxium-install-UCM-files.sh: Installation of UCM finished 
# wrapper-linuxium-install-broadcom-drivers.sh
./linuxium-install-broadcom-drivers.sh: Extracting Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Reloading Broadcom driver ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom bluetooth service ...
Created symlink /etc/systemd/system/multi-user.target.wants/brcmbt.service -> /lib/systemd/system/brcmbt.service.
./linuxium-install-broadcom-drivers.sh: Starting Broadcom bluetooth service ...
Running in chroot, ignoring request.
./linuxium-install-broadcom-drivers.sh: Installation of Broadcom finished 
Persistence partition of '200MB' added ...
Respun ISO created as 'linuxium-persistence-030817-artful-desktop-amd64.iso'.

The new flags are only available from a CLI invocation:


and the upgraded script can be downloaded from 'isorespin.sh'.

Please donate if you find the script useful using the following link http://goo.gl/nXWSGf.

13 comments:

Jean Laganiere said...

Isorespin.sh seems to work for me. It does produce an ISO. But should I worry about those errors in the log file (my hostname is "a" and it does resolve from the /etc/hosts file)?
...
Command run ...
# wrapper-linuxium-install-UCM-files.sh
./linuxium-install-UCM-files.sh: Extracting UCM files ...
sudo: unable to resolve host a: No such file or directory
sudo: unable to resolve host a: No such file or directory
sudo: unable to resolve host a: No such file or directory
sudo: unable to resolve host a: No such file or directory
./linuxium-install-UCM-files.sh: Installing UCM files ...
sudo: unable to resolve host a: No such file or directory
sudo: unable to resolve host a: No such file or directory
...

Thanks for your excellent work!

Linuxium said...

No it does not matter about those messages. In the past you would edit the 'wrapper-linuxium-install-UCM-files.sh' script and change the hostname from 'LINUXIUMONE' to your machine name. Now the script gets automatically downloaded I need to look at modifying it to prevent this issue.

Jean Laganiere said...

Thanks Ian, I got rid of those massages by renaming my machine 'LINUXIUMONE'. :)

Cheers.

Linuxium said...

I've just uploaded new versions of the two 'wrapper' scripts that should remove this issue and work regardless of your machine name.

Linuxium said...

Ubuntu is really too resource demanding for the STCK1A8FLC device and I'd recommend going for a lighter distro like Lubuntu or similar.

Amanulla.S said...

hi Linuxium,

cheers for your Great efforts and your continuous updates in this topic and works.

I really want to try backbox in my asus x205ta.

because of working in some projects,I'll try to spin by own on later.
but, I request you to spin it for me and share backbox ISO for download and try..

Kindly consider this and give me positive confirmation.


Thanks in Advance .


Linuxium said...

It will be faster for you to respin the ISO than wait for my upload!

MaxCs said...

Hi Linuxium,
thank you for sharing your work.

I'd like to use linux on an Atom Z8300 device.
I used your script succesfully with the "-i lubuntu-17.04-desktop-amd64.iso --atom" options/switchs.

Because the wifi (Broadcom BCMSDH43XX) doesn't work I tryed to add the "--update" option/switch but the script exit with "Cannot find mainline kernel."

First of all, I'm not sure if the new kernel colud resolve the wifi issue, or if is still needed to use old scripts to add some drivers in order to make that hardware to work.

If the new kernel is the solution, what can be the cause of the "Cannot find mainline kernel." message ? The kernel related files are downloaded during the process but never moved in iso-chroot/boot/ directory, so the variables MAINLINE_INITRD and MANINLINE_VMLINUZ are empty.

Thank you.
Best regards.
MaxCs

Linuxium said...

When you get an error can you provide the full command and error messages? Without them it is very difficult to debug when the problem cannot be reproduced as in this case.

I'd recommend you re-download my 'isorespin.sh' script again (to make sure you have the latest version) and run with the options (including both with or without '--update') as required. For a Cherry Trail device I'd use the '--update' option to get the latest kernel with all the patches that are suitable for the Cherry Trail SoC.

In terms of fixing the wifi, what device are you running on and have you copied over the wifi firmware (see 'Wifi issues' on 'https://linuxiumcomau.blogspot.com.au/2017/06/customizing-ubuntu-isos-documentation.html')?

Sin said...

Hello
First of all thank só much for your efforts!! It is an amazing work you are doing here.
When i try to install your ISO i get the following:
"The 'grub-efi-ia-32' package failed to install into /target/." I am using an intel stick from a spanish brand Z7357F unfortunatelly i am blocked it can detect the WiFi networks but does not connect. Do you havê any advice for me?

Linuxium said...

Unfortunately and as mentioned on 'http://linuxiumcomau.blogspot.com.au/2017/06/customizing-ubuntu-isos-documentation.html' success in installing GRUB packages without the internet on 32-bit devices depends on the original ISO installation capability. For example as Lubuntu 17.04 cannot be installed on 64-bit devices without internet as the ISO does not contain '/pool' with required packages so likewise after respinning its installation on 32-bit devices is not possible. However Lubuntu 16.04.2 can be installed on both 64-bit and 32-bit devices after respinning. The best solution for you would be to use an ethernet USB dongle if your wifi is blocked.

MaxCs said...

Hi Linuxium,
thanks for the response.

At the end the 2 commands used against your last script (7.3.2).

Thank you very much.
Best regards.
MaxCs


./isorespin.sh -i lubuntu-17.04-desktop-amd64.iso --atom --update
Extracting ISO ...
Parallel unsquashfs: Using 4 processors
113162 inodes (117611 blocks) to write

[======/] 117611/117611 100%

created 89645 files
created 13800 directories
created 23483 symlinks
created 7 devices
created 0 fifos
Extracting isorespin files ...
Updating bootloader/bootmanager ...
Fetching mainline kernel packages ...
Installing mainline kernel packages ...
./isorespin.sh: Cannot find mainline kernel.








./isorespin.sh -i lubuntu-17.04-desktop-amd64.iso --atom
Extracting ISO ...
Parallel unsquashfs: Using 4 processors
113162 inodes (117611 blocks) to write

[=======|] 117611/117611 100%

created 89645 files
created 13800 directories
created 23483 symlinks
created 7 devices
created 0 fifos
Extracting isorespin files ...
Updating bootloader/bootmanager ...
Installing local packages ...
Adding files/directories ...
Running commands ...
cat: iso-chroot/root/.command.log: No such file or directory
cat: iso-chroot/root/.command.log: No such file or directory
Spinning ISO ...
chroot: failed to run command 'dpkg-query': Exec format error
Parallel mksquashfs: Using 4 processors
Creating 4.0 filesystem on iso-directory-structure/casper/filesystem.squashfs, block size 131072.
[======-] 94130/94130 100%

Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
compressed data, compressed metadata, compressed fragments, compressed xattrs
duplicates are removed
Filesystem size 887035.95 Kbytes (866.25 Mbytes)
39.55% of uncompressed filesystem size (2242754.04 Kbytes)
Inode table size 1215734 bytes (1187.24 Kbytes)
26.79% of uncompressed inode table size (4538618 bytes)
Directory table size 1231333 bytes (1202.47 Kbytes)
40.33% of uncompressed directory table size (3052940 bytes)
Number of duplicate files found 12736
Number of inodes 126940
Number of files 89650
Number of fragments 5553
Number of symbolic links 23483
Number of device nodes 7
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 13800
Number of ids (unique uids + gids) 24
Number of uids 6
root (0)
unknown (113)
man (6)
_apt (105)
lightdm (108)
syslog (104)
Number of gids 20
root (0)
dip (30)
shadow (42)
ntp (114)
dialout (20)
utmp (43)
tty (5)
crontab (107)
uuidd (111)
ssh (117)
netdev (109)
staff (50)
bluetooth (118)
man (12)
scanner (119)
syslog (108)
whoopsie (115)
lp (7)
adm (4)
mail (8)
xorriso 1.4.2 : RockRidge filesystem manipulator, libburnia project.

Drive current: -outdev 'stdio:../../linuxium-atom-lubuntu-17.04-desktop-amd64.iso'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data, 58.3g free
xorriso : WARNING : -volid text problematic as automatic mount point name
xorriso : WARNING : -volid text does not comply to ISO 9660 / ECMA 119 rules
Added to ISO image: directory '/'='/home/user/Downloads/isorespin/iso-directory-structure'
xorriso : UPDATE : 607 files added in 1 seconds
xorriso : UPDATE : 607 files added in 1 seconds
xorriso : NOTE : Copying to System Area: 512 bytes from file '/home/user/Downloads/isorespin/isohdpfx.bin'
libisofs: NOTE : Aligned image size to cylinder size by 493 blocks
xorriso : UPDATE : 5.26% done
xorriso : UPDATE : 97.39% done
ISO image produced: 467968 sectors
Written to medium : 467968 sectors at LBA 0
Writing to 'stdio:../../linuxium-atom-lubuntu-17.04-desktop-amd64.iso' completed successfully.

./isorespin.sh: Respun ISO created as 'linuxium-atom-lubuntu-17.04-desktop-amd64.iso' ... see logfile 'isorespin.log' for details.

Linuxium said...

I can't replicate the problem. Can you try again as the only thing I can think of is that the mainline kernel repository went offline while you were re-spinning.

Post a Comment

All comments now moderated so that spam can be deleted.