Sunday, 6 August 2017

Respinning security distros and upgrading packages

A while back I dropped Debian and Debian based ISO support from my 'isorespin.sh' script as the release of Debian 9 Stretch uses a v4.9 kernel rather than a v3.16 kernel meaning that the kernel cannot be upgraded with Canonical's HDMI and RTL8723BS DKMS support. I also dropped their support because I do not agree with using a kernel compiled for one distro's userland with a different distro's userland as for example in using an Ubuntu kernel to boot a Fedora ISO.

This meant no more respinning Kali ISOs but since I'm again receiving requests for its support it got me thinking about what Ubuntu-based security distros existed. So following some research I've added support for BackBox Linux a 'penetration testing and security assessment oriented Linux distribution providing a network and systems analysis toolkit'.


Respinning is simple using my latest version of 'isorespin.sh':

Script '/usr/local/bin/isorespin.sh' called with '-i backbox-5-amd64.iso --atom --update' ...
Work directory 'isorespin' used ...
ISO '/home/linuxium/backbox-5-amd64.iso' respun ...
Bootloader 'GRUB' added ...
Kernel updated with mainline kernel version '4.13.0-041300rc3-generic' ...
Local package '/home/linuxium/isorespin/rtl8723bs_4.12.0_amd64.deb' added ...
File '/home/linuxium/isorespin/linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/linuxium-install-broadcom-drivers.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-broadcom-drivers.sh' added ...
Command run ...
# wrapper-linuxium-install-UCM-files.sh
./linuxium-install-UCM-files.sh: Extracting UCM files ...
./linuxium-install-UCM-files.sh: Installing UCM files ...
./linuxium-install-UCM-files.sh: Reloading UCM driver ...
./linuxium-install-UCM-files.sh: Installation of UCM finished 
# wrapper-linuxium-install-broadcom-drivers.sh
./linuxium-install-broadcom-drivers.sh: Extracting Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Reloading Broadcom driver ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom bluetooth service ...
Created symlink /etc/systemd/system/multi-user.target.wants/brcmbt.service, pointing to /lib/systemd/system/brcmbt.service.
./linuxium-install-broadcom-drivers.sh: Starting Broadcom bluetooth service ...
Running in chroot, ignoring request.
./linuxium-install-broadcom-drivers.sh: Installation of Broadcom finished 
Respun ISO created as 'linuxium-v4.13-rc3-backbox-5-amd64.iso'.

I've also had another look at Kali as whilst their official ISOs use a Debian kernel they also offer Kali Metapackages which 'give you the flexibility to install specific subsets of tools based on your particular needs'. Following the documented instructions I looked at how I could update my script to allow the addition of these metapackages when respinning. As a result I've added a new option '--key' to add GPG keys to the APT keyring allowing packages to be downloaded from signed repositories. It is now possible to respin an Ubuntu ISO adding the packages:


There are some restrictions/limitations. Unity isn't supported and I've found adding a GPG key to a 17.04 or 17.10 release fails. Additionally adding the 'kali-linux-full' package results in dependency issues. However it is possible to respin the recently released Ubuntu GNOME 16.04.3 and add 'kali-linux' and 'kali-linux-top10':

Script '/usr/local/bin/isorespin.sh' called with '-i ubuntu-gnome-16.04.3-desktop-amd64.iso --atom -u --key adv --keyserver keyserver.ubuntu.com --recv-keys ED444FF07D8D0BF6 --repository deb http://http.kali.org/kali kali-rolling main contrib non-free -p kali-linux -p kali-linux-top10' ...
Work directory 'isorespin' used ...
ISO '/home/linuxium/ubuntu-gnome-16.04.3-desktop-amd64.iso' respun ...
Bootloader 'GRUB' added ...
Kernel updated with mainline kernel version '4.13.0-041300rc3-generic' ...
Key 'adv --keyserver keyserver.ubuntu.com --recv-keys ED444FF07D8D0BF6' added ...
Repository 'deb http://http.kali.org/kali kali-rolling main contrib non-free' added ...
Package 'kali-linux' added ...
Package 'kali-linux-top10' added ...
Local package '/home/linuxium/isorespin/rtl8723bs_4.12.0_amd64.deb' added ...
File '/home/linuxium/isorespin/linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/linuxium-install-broadcom-drivers.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-broadcom-drivers.sh' added ...
Command run ...
# wrapper-linuxium-install-UCM-files.sh
./linuxium-install-UCM-files.sh: Extracting UCM files ...
./linuxium-install-UCM-files.sh: Installing UCM files ...
./linuxium-install-UCM-files.sh: Reloading UCM driver ...
./linuxium-install-UCM-files.sh: Installation of UCM finished 
# wrapper-linuxium-install-broadcom-drivers.sh
./linuxium-install-broadcom-drivers.sh: Extracting Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Reloading Broadcom driver ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom bluetooth service ...
Created symlink /etc/systemd/system/multi-user.target.wants/brcmbt.service, pointing to /lib/systemd/system/brcmbt.service.
./linuxium-install-broadcom-drivers.sh: Starting Broadcom bluetooth service ...
Running in chroot, ignoring request.
./linuxium-install-broadcom-drivers.sh: Installation of Broadcom finished 
Respun ISO created as 'linuxium-v4.13-rc3-ubuntu-gnome-16.04.3-desktop-amd64.iso'.

Lubuntu is also supported:


and by adding 'kali-desktop-lxde' additional LXDE packages are included (note 'Other'):


Script '/usr/local/bin/isorespin.sh' called with '-i lubuntu-16.04.3-desktop-amd64.iso --atom -u --key adv --keyserver keyserver.ubuntu.com --recv-keys ED444FF07D8D0BF6 --repository deb http://http.kali.org/kali kali-rolling main contrib non-free -p kali-linux -p kali-desktop-lxde -p kali-linux-top10' ...
Work directory 'isorespin' used ...
ISO '/home/linuxium/lubuntu-16.04.3-desktop-amd64.iso' respun ...
Bootloader 'GRUB' added ...
Kernel updated with mainline kernel version '4.13.0-041300rc3-generic' ...
Key 'adv --keyserver keyserver.ubuntu.com --recv-keys ED444FF07D8D0BF6' added ...
Repository 'deb http://http.kali.org/kali kali-rolling main contrib non-free' added ...
Package 'kali-linux' added ...
Package 'kali-desktop-lxde' added ...
Package 'kali-linux-top10' added ...
Local package '/home/linuxium/isorespin/rtl8723bs_4.12.0_amd64.deb' added ...
File '/home/linuxium/isorespin/linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/linuxium-install-broadcom-drivers.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-broadcom-drivers.sh' added ...
Command run ...
# wrapper-linuxium-install-UCM-files.sh
./linuxium-install-UCM-files.sh: Extracting UCM files ...
./linuxium-install-UCM-files.sh: Installing UCM files ...
./linuxium-install-UCM-files.sh: Reloading UCM driver ...
./linuxium-install-UCM-files.sh: Installation of UCM finished 
# wrapper-linuxium-install-broadcom-drivers.sh
./linuxium-install-broadcom-drivers.sh: Extracting Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Reloading Broadcom driver ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom bluetooth service ...
Created symlink /etc/systemd/system/multi-user.target.wants/brcmbt.service, pointing to /lib/systemd/system/brcmbt.service.
./linuxium-install-broadcom-drivers.sh: Starting Broadcom bluetooth service ...
Running in chroot, ignoring request.
./linuxium-install-broadcom-drivers.sh: Installation of Broadcom finished 
Respun ISO created as 'linuxium-v4.13-rc3-lubuntu-16.04.3-desktop-amd64.iso'.

Finally I've added another option '--upgrade' which simply performs an 'apt-get upgrade' on the ISO's packages. So for example having downloaded an Artful daily ISO, I can respin it with the latest packages:


Script '/usr/local/bin/isorespin.sh' called with '-i 030817-artful-desktop-amd64.iso --upgrade --rolling-unstable --atom -s 200MB' ...
Work directory 'isorespin' used ...
ISO '/home/linuxium/030817-artful-desktop-amd64.iso' respun ...
Kernel boot parameters 'persistent' added ...
Bootmanager 'rEFInd' added ...
Distro upgraded ...
Package 'linux-headers-4.12.0-9 linux-headers-4.12.0-9-generic linux-image-4.12.0-9-generic linux-image-extra-4.12.0-9-generic' added ...
Local package '/home/linuxium/isorespin/rtl8723bt_4.12.0_amd64.deb' added ...
File '/home/linuxium/isorespin/linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-UCM-files.sh' added ...
File '/home/linuxium/isorespin/linuxium-install-broadcom-drivers.sh' added ...
File '/home/linuxium/isorespin/wrapper-linuxium-install-broadcom-drivers.sh' added ...
Command run ...
# wrapper-linuxium-install-UCM-files.sh
./linuxium-install-UCM-files.sh: Extracting UCM files ...
./linuxium-install-UCM-files.sh: Installing UCM files ...
./linuxium-install-UCM-files.sh: Reloading UCM driver ...
./linuxium-install-UCM-files.sh: Installation of UCM finished 
# wrapper-linuxium-install-broadcom-drivers.sh
./linuxium-install-broadcom-drivers.sh: Extracting Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom files ...
./linuxium-install-broadcom-drivers.sh: Reloading Broadcom driver ...
./linuxium-install-broadcom-drivers.sh: Installing Broadcom bluetooth service ...
Created symlink /etc/systemd/system/multi-user.target.wants/brcmbt.service -> /lib/systemd/system/brcmbt.service.
./linuxium-install-broadcom-drivers.sh: Starting Broadcom bluetooth service ...
Running in chroot, ignoring request.
./linuxium-install-broadcom-drivers.sh: Installation of Broadcom finished 
Persistence partition of '200MB' added ...
Respun ISO created as 'linuxium-persistence-030817-artful-desktop-amd64.iso'.

The new flags are only available from a CLI invocation:


and the upgraded script can be downloaded from 'isorespin.sh'.

Please donate if you find the script useful using the following link http://goo.gl/nXWSGf.

14 comments:

  1. Isorespin.sh seems to work for me. It does produce an ISO. But should I worry about those errors in the log file (my hostname is "a" and it does resolve from the /etc/hosts file)?
    ...
    Command run ...
    # wrapper-linuxium-install-UCM-files.sh
    ./linuxium-install-UCM-files.sh: Extracting UCM files ...
    sudo: unable to resolve host a: No such file or directory
    sudo: unable to resolve host a: No such file or directory
    sudo: unable to resolve host a: No such file or directory
    sudo: unable to resolve host a: No such file or directory
    ./linuxium-install-UCM-files.sh: Installing UCM files ...
    sudo: unable to resolve host a: No such file or directory
    sudo: unable to resolve host a: No such file or directory
    ...

    Thanks for your excellent work!

    ReplyDelete
    Replies
    1. No it does not matter about those messages. In the past you would edit the 'wrapper-linuxium-install-UCM-files.sh' script and change the hostname from 'LINUXIUMONE' to your machine name. Now the script gets automatically downloaded I need to look at modifying it to prevent this issue.

      Delete
    2. Thanks Ian, I got rid of those massages by renaming my machine 'LINUXIUMONE'. :)

      Cheers.

      Delete
    3. I've just uploaded new versions of the two 'wrapper' scripts that should remove this issue and work regardless of your machine name.

      Delete
  2. First of all, let me thanks for all your effort and its really helpful all your posting.
    I have STCK1A8FLC (5 gigi usage & 1 RAM). I'm using your ISO linuxium-persistence-030817-artful-desktop-amd64.iso and craeted USB bootable using RUFUS tool. I booted and taking tooo much time to install (3 hours) and crashed while configuring hardware and very disappointed.
    Is there any mistake i'm doing? Please clarify and I really appreciate that. Thanks.

    ReplyDelete
    Replies
    1. Ubuntu is really too resource demanding for the STCK1A8FLC device and I'd recommend going for a lighter distro like Lubuntu or similar.

      Delete
  3. hi Linuxium,

    cheers for your Great efforts and your continuous updates in this topic and works.

    I really want to try backbox in my asus x205ta.

    because of working in some projects,I'll try to spin by own on later.
    but, I request you to spin it for me and share backbox ISO for download and try..

    Kindly consider this and give me positive confirmation.


    Thanks in Advance .


    ReplyDelete
    Replies
    1. It will be faster for you to respin the ISO than wait for my upload!

      Delete
  4. Hi Linuxium,
    thank you for sharing your work.

    I'd like to use linux on an Atom Z8300 device.
    I used your script succesfully with the "-i lubuntu-17.04-desktop-amd64.iso --atom" options/switchs.

    Because the wifi (Broadcom BCMSDH43XX) doesn't work I tryed to add the "--update" option/switch but the script exit with "Cannot find mainline kernel."

    First of all, I'm not sure if the new kernel colud resolve the wifi issue, or if is still needed to use old scripts to add some drivers in order to make that hardware to work.

    If the new kernel is the solution, what can be the cause of the "Cannot find mainline kernel." message ? The kernel related files are downloaded during the process but never moved in iso-chroot/boot/ directory, so the variables MAINLINE_INITRD and MANINLINE_VMLINUZ are empty.

    Thank you.
    Best regards.
    MaxCs

    ReplyDelete
    Replies
    1. When you get an error can you provide the full command and error messages? Without them it is very difficult to debug when the problem cannot be reproduced as in this case.

      I'd recommend you re-download my 'isorespin.sh' script again (to make sure you have the latest version) and run with the options (including both with or without '--update') as required. For a Cherry Trail device I'd use the '--update' option to get the latest kernel with all the patches that are suitable for the Cherry Trail SoC.

      In terms of fixing the wifi, what device are you running on and have you copied over the wifi firmware (see 'Wifi issues' on 'https://linuxiumcomau.blogspot.com.au/2017/06/customizing-ubuntu-isos-documentation.html')?

      Delete
    2. Hi Linuxium,
      thanks for the response.

      At the end the 2 commands used against your last script (7.3.2).

      Thank you very much.
      Best regards.
      MaxCs


      ./isorespin.sh -i lubuntu-17.04-desktop-amd64.iso --atom --update
      Extracting ISO ...
      Parallel unsquashfs: Using 4 processors
      113162 inodes (117611 blocks) to write

      [======/] 117611/117611 100%

      created 89645 files
      created 13800 directories
      created 23483 symlinks
      created 7 devices
      created 0 fifos
      Extracting isorespin files ...
      Updating bootloader/bootmanager ...
      Fetching mainline kernel packages ...
      Installing mainline kernel packages ...
      ./isorespin.sh: Cannot find mainline kernel.








      ./isorespin.sh -i lubuntu-17.04-desktop-amd64.iso --atom
      Extracting ISO ...
      Parallel unsquashfs: Using 4 processors
      113162 inodes (117611 blocks) to write

      [=======|] 117611/117611 100%

      created 89645 files
      created 13800 directories
      created 23483 symlinks
      created 7 devices
      created 0 fifos
      Extracting isorespin files ...
      Updating bootloader/bootmanager ...
      Installing local packages ...
      Adding files/directories ...
      Running commands ...
      cat: iso-chroot/root/.command.log: No such file or directory
      cat: iso-chroot/root/.command.log: No such file or directory
      Spinning ISO ...
      chroot: failed to run command 'dpkg-query': Exec format error
      Parallel mksquashfs: Using 4 processors
      Creating 4.0 filesystem on iso-directory-structure/casper/filesystem.squashfs, block size 131072.
      [======-] 94130/94130 100%

      Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
      compressed data, compressed metadata, compressed fragments, compressed xattrs
      duplicates are removed
      Filesystem size 887035.95 Kbytes (866.25 Mbytes)
      39.55% of uncompressed filesystem size (2242754.04 Kbytes)
      Inode table size 1215734 bytes (1187.24 Kbytes)
      26.79% of uncompressed inode table size (4538618 bytes)
      Directory table size 1231333 bytes (1202.47 Kbytes)
      40.33% of uncompressed directory table size (3052940 bytes)
      Number of duplicate files found 12736
      Number of inodes 126940
      Number of files 89650
      Number of fragments 5553
      Number of symbolic links 23483
      Number of device nodes 7
      Number of fifo nodes 0
      Number of socket nodes 0
      Number of directories 13800
      Number of ids (unique uids + gids) 24
      Number of uids 6
      root (0)
      unknown (113)
      man (6)
      _apt (105)
      lightdm (108)
      syslog (104)
      Number of gids 20
      root (0)
      dip (30)
      shadow (42)
      ntp (114)
      dialout (20)
      utmp (43)
      tty (5)
      crontab (107)
      uuidd (111)
      ssh (117)
      netdev (109)
      staff (50)
      bluetooth (118)
      man (12)
      scanner (119)
      syslog (108)
      whoopsie (115)
      lp (7)
      adm (4)
      mail (8)
      xorriso 1.4.2 : RockRidge filesystem manipulator, libburnia project.

      Drive current: -outdev 'stdio:../../linuxium-atom-lubuntu-17.04-desktop-amd64.iso'
      Media current: stdio file, overwriteable
      Media status : is blank
      Media summary: 0 sessions, 0 data blocks, 0 data, 58.3g free
      xorriso : WARNING : -volid text problematic as automatic mount point name
      xorriso : WARNING : -volid text does not comply to ISO 9660 / ECMA 119 rules
      Added to ISO image: directory '/'='/home/user/Downloads/isorespin/iso-directory-structure'
      xorriso : UPDATE : 607 files added in 1 seconds
      xorriso : UPDATE : 607 files added in 1 seconds
      xorriso : NOTE : Copying to System Area: 512 bytes from file '/home/user/Downloads/isorespin/isohdpfx.bin'
      libisofs: NOTE : Aligned image size to cylinder size by 493 blocks
      xorriso : UPDATE : 5.26% done
      xorriso : UPDATE : 97.39% done
      ISO image produced: 467968 sectors
      Written to medium : 467968 sectors at LBA 0
      Writing to 'stdio:../../linuxium-atom-lubuntu-17.04-desktop-amd64.iso' completed successfully.

      ./isorespin.sh: Respun ISO created as 'linuxium-atom-lubuntu-17.04-desktop-amd64.iso' ... see logfile 'isorespin.log' for details.

      Delete
    3. I can't replicate the problem. Can you try again as the only thing I can think of is that the mainline kernel repository went offline while you were re-spinning.

      Delete
  5. Hello
    First of all thank só much for your efforts!! It is an amazing work you are doing here.
    When i try to install your ISO i get the following:
    "The 'grub-efi-ia-32' package failed to install into /target/." I am using an intel stick from a spanish brand Z7357F unfortunatelly i am blocked it can detect the WiFi networks but does not connect. Do you havê any advice for me?

    ReplyDelete
    Replies
    1. Unfortunately and as mentioned on 'http://linuxiumcomau.blogspot.com.au/2017/06/customizing-ubuntu-isos-documentation.html' success in installing GRUB packages without the internet on 32-bit devices depends on the original ISO installation capability. For example as Lubuntu 17.04 cannot be installed on 64-bit devices without internet as the ISO does not contain '/pool' with required packages so likewise after respinning its installation on 32-bit devices is not possible. However Lubuntu 16.04.2 can be installed on both 64-bit and 32-bit devices after respinning. The best solution for you would be to use an ethernet USB dongle if your wifi is blocked.

      Delete