How to create an ISO that mimics the installed packages from an ISO or those locally installed using 'isomimicpkgs.sh'

This is the second of three posts introducing a couple of new 'ISO' tools that I've developed to complement my 'isorespin.sh' script.

The second tool is 'isomimicpkgs.sh' which is a script to compare installed packages in two Ubuntu (or Ubuntu flavour), Linux Mint, neon, elementary, BackBox or Peppermint desktop ISOs or Ubuntu live server ISOs, or one such ISO with locally installed packages and identify which packages need to be added to the second ISO to mimic the first ISO or added to the first ISO to mimic those locally installed. In otherwords it will show how to update an ISO with either the packages found in another ISO or with those locally installed.

For example, say you want to create a test enviroment with the same packages as those locally installed. Normally you would install the same Ubuntu release ISO and then install the packages one by one in order to replicate the enviroment. However this is not so simple when you have a lot of packages installed.

You can start by finding the ISO that was used for the initial install by looking at the contents of the file '/var/log/installer/media-info':

linuxium@LINUXIUM:~$ cat /var/log/installer/media-info 
Ubuntu 18.04 LTS "Bionic Beaver" - Release amd64 (20180426)linuxium@LINUXIUM:~$ 
linuxium@LINUXIUM:~$

Next you can search through the 'apt' history files looking for entries showing an application installation:

See how the highlighted entry shows the package installed together with the dependent packages that were automatically installed at the same time.

However this will not show any packages that were installed using 'dpkg' or by a tool like 'gdebi'. To see all the packages we can look at '/var/log/dpkg.log' however the 'package -> dependent package' relationship is somewhat lost:

Finally these logs do not last for ever as they are rotated to save space as defined by their respective configuration files:

So one solution to the problem of creating a duplicate test environment would be to use 'isocomparepkgs.sh' to establish which packages were locally installed and not part of the original ISO and then respin the original ISO adding these packages. Except that in a more complex environment this can become a somewhat unrealistic alternative :

Which finally leads me to my other tool: 'isomimicpkgs.sh'.

        linuxium@LINUXIUM:~$ isomimicpkgs.sh

        Usage: isomimicpkgs.sh <target ISO>
           or: isomimicpkgs.sh <source ISO> <target ISO>

        linuxium@LINUXIUM:~$

The usage syntax is straightforward: you either run the script against two ISOs to see how to mimic the package of those installed in the source ISO to the target ISO or you run the script against an ISO to see how to mimic the locally installed packages to that ISO.

In the example we have been using so far you would run the command:

isomimicpkgs.sh ubuntu-18.04-desktop-amd64.iso

The resultant output looks like this:

The tool first identifies those local packages that need to be installed manually as they are not Canonical packages (e.g. 'google-chrome-stable', 'megasync' and 'xnview').

The tool has then simplified the number of packages that were needed to be installed down from 698 to 102 by attemting to reverse engineer the package dependencies and determine a minimum set of packages that obtains the same result.

However as a result it has highlighted some packages that then also need to be removed (or purged), either because they had been previously removed since original installation (for example 'youtube-dl') or due to a dependency issue (in this case it is actually 'php7.2-gd'). Therefore rather than simply removing all the identified additional packages as there could be consequencies because of package dependencies it is advisable to check each one individually first. This is best achieved by respinning the intermediate ISO using the '--interactive' option. For example:

The resultant ISO can then be compared with the locally installed packages to verify that we have successfully mimicked the local environment:

In the final post of this series I will demonstrate the usage of my 'ISO' tools in a real-life example.

The initial release of 'isomimicpkgs.sh' can be downloaded from here.

Please donate if you find the script useful using the following link http://goo.gl/nXWSGf as everything helps with development costs.

Comparing packages between ISOs or between an ISO and those locally installed using 'isocomparepkgs.sh'

This is the first of three posts introducing a couple of new 'ISO' tools that I've developed to complement my 'isorespin.sh' script.

The first tool is 'isocomparepkgs.sh' which is a script to compare installed packages between two Ubuntu (or Ubuntu flavour), Linux Mint, neon, elementary, BackBox or Peppermint desktop ISOs or Ubuntu live server ISOs or between one such ISO and the locally installed packages on the system where the script is run.

linuxium@LINUXIUM:~$ isocomparepkgs.sh
Usage: isocomparepkgs.sh <ISO> [<ISO>]
linuxium@LINUXIUM:~$

The usage syntax is straightforward: you either run the script against two ISOs to get a package comparison of those installed in each or you run the script against one ISO to get a comparison of packages in the ISO verses those locally installed.

For example, say you want to see the package difference between the original Ubuntu 18.04 ISO and the latest point release of 18.04.4. Simply enter the command:

isocomparepkgs.sh ubuntu-18.04-desktop-amd64.iso ubuntu-18.04.4-desktop-amd64.iso

The resultant output looks like this:

with the logfile 'isocomparepkgs.log' also including a side-by-side comparison:

It is important to note that the script only shows the difference between installed packages based on their name and not their version or release. Also it only shows packages and not userspace file differences. 

So the difference between a respun Ubuntu 18.04.4 ISO using the 'atom' option and the original ISO only shows the additional bluetooth package and not the wifi and ALSA UCM files which were also added as a result of respinning:

When comparing packages in an ISO with those locally installed it is important to remember than an ISO also includes several packages used only as part of the installation process:

and that the list of locally installed packages also includes all the dependent packages which can make it difficult to determine the actual packages used in the install commands:

which is a why the second post in this series may be of interest.

The initial release of 'isocomparepkgs.sh' can be downloaded from here.

Please donate if you find the script useful using the following link http://goo.gl/nXWSGf as everything helps with development costs.