Merge branch 'main' into arm-support

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,49 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: bug
+assignees: ''
+
+---
+
+<!--
+Before making a bug report please check that:
+- The USB drive/SD card you are using isn't faulty
+  - Dirt cheap USB 2.0 drives or fake high capacity ones will not work
+  - Generally if your drive is unbranded it is slow and of poor quality
+- The disk image you are using is not corrupted
+- You have *some* knowledge on how to use Linux
+-->
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots / Photos**
+If applicable, add screenshots or photos to help explain your problem.
+
+If you are reporting an issue with the build process, please run the scripts in debug mode by putting `DEBUG=1` before the build commmand, like `sudo DEBUG=1 ./build_complete.sh`.
+
+**Target Chrome OS Device (please complete the following information):**
+ - Board Name (e.g. dedede)
+ - Device Name (e.g. drawcia) <!-- this page has a list of them: https://chromiumdash.appspot.com/serving-builds -->
+ - Shimboot version (e.g. v1.0.1) 
+
+<!-- If you are using a prebuilt image please make note of that and delete this section. -->
+<!-- This section is for the device you build shimboot on, not your chromebook. -->
+**Build Device (please complete the following information):**
+ - OS: [e.g. Debian 12]
+ - Neofetch out [run `neofetch --stdout` and paste it here]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/device-compatibility-report.md

@@ -0,0 +1,24 @@
+---
+name: Device Compatibility Report
+about: Report your experiences using Shimboot on a previously untested device.
+title: 'Device compatibility report for: (device name)'
+labels: documentation
+assignees: ''
+
+---
+
+This template is not meant to be used if Shimboot fails completely. Submit a bug report instead.
+
+**Compatibility Info**:
+- Board Name: (e.g. octopus)
+- X11: yes/no/untested  <!-- X11 is what handles your system's GUI. If a desktop showed up at all then X11 worked. -->
+- Wifi: yes/no/untested 
+- Internal Audio: yes/no/untested <!-- This is referring to your laptop speakers. -->
+- Backlight: yes/no/untested  <!-- This means whether or not you can control your screen brightness. -->
+- Touchscreen: yes/no/untested
+- 3D acceleration: yes/no/untested <!-- Does `glxinfo | grep "renderer string"` report the correct GPU name? -->
+- Bluetooth: yes/no/untested
+- Webcam: yes/no/untested
+
+
+**Other Notes:**:

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: enhancement
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/workflows/build-image.yaml

@@ -9,10 +9,19 @@ jobs:
 
     strategy: 
       matrix:
-        board: [dedede, octopus, coral, grunt]
+        board: [dedede, octopus, coral, grunt, nissa, zork]
 
     runs-on: ubuntu-latest
     steps:
+      - name: Maximize build space
+        uses: easimon/maximize-build-space@master
+        with:
+          root-reserve-mb: 4096
+          swap-size-mb: 1024
+          remove-dotnet: 'true'
+          remove-android: 'true'
+          remove-haskell: 'true'
+
       - name: download repo
         uses: actions/checkout@v4
       
@@ -24,7 +33,7 @@ jobs:
         with:
           name: shimboot_${{ matrix.board }}
           path: data/shimboot_${{ matrix.board }}.bin
-          compression-level: 9
+          compression-level: 7
   
       - name: create release
         uses: softprops/action-gh-release@v1

README.md

@@ -2,15 +2,26 @@
 
 Shimboot is a collection of scripts for patching a Chrome OS RMA shim to serve as a bootloader for a standard Linux distribution. It allows you to boot a full desktop Debian install on a Chromebook, without needing to unenroll it or modify the firmware.
 
+## Features:
+- Run a full Debian installation on a Chromebook
+- Does not modify the firmware
+- Works on enterprise enrolled devices
+- Can boot Chrome OS with no restrictions (useful for enrolled devices)
+- Nearly full device compatibility
+- Optional disk compression 
+- Multiple desktop environments supported
+
 ## About:
 Chrome OS RMA shims are bootable disk images which are designed to run a variety of diagnostic utilities on Chromebooks, and they'll work even if the device is enterprise enrolled. Unfortunately for Google, there exists a [security flaw](https://sh1mmer.me/) where the root filesystem of the RMA shim is not verified. This lets us replace the rootfs with anything we want, including a full Linux distribution.
 
-Simply replacing the shim's rootfs doesn't work, as it boots in an environment friendly to the RMA shim, not regular Linux distros. To get around this, a separate bootloader is required to transition from the shim environment to the main rootfs. This bootloader then does `pivot_root` to enter the rootfs, where it then starts the init system.
+Simply replacing the shim's rootfs doesn't work, as it boots in an environment friendly to the RMA shim, not regular Linux distros. To get around this, a separate bootloader is required to transition from the shim environment to the main rootfs. This bootloader then runs `pivot_root` to enter the rootfs, where it then starts the init system.
 
 Another problem is encountered at this stage: the Chrome OS kernel will complain about systemd's mounts, and the boot process will hang. A simple workaround is to [apply a patch](https://github.com/ading2210/chromeos-systemd) to systemd, and then it can be recompiled and hosted at a [repo somewhere](https://github.com/ading2210/shimboot-repo).
 
 After copying all the firmware from the recovery image and shim to the rootfs, we're able to boot to a mostly working XFCE desktop.
 
+The main advantages of this approach are that you don't need to touch the device's firmware in order to run Linux. Simply rebooting and unplugging the USB drive will return the device to normal, which can be useful if the device is enterprise enrolled. However, since we are stuck with the kernel from the RMA shim, some features such as audio and suspend may not work.
+
 ### Partition Layout:
 1. 1MB dummy stateful partition
 2. 32MB Chrome OS kernel
@@ -20,56 +31,51 @@ After copying all the firmware from the recovery image and shim to the rootfs, w
 Note that rootfs partitions have to be named `shimboot_rootfs:<partname>` for the bootloader to recognize them.
 
 ## Status:
-Driver support depends on the device you are using shimboot on. This list is for the [`dedede`](https://chrome100.dev/board/dedede/) board, which is the only device I was able to do extensive testing on. The `patch_rootfs.sh` script attempts to copy all the firmware from the shim and recovery image into the rootfs, so expect most things to work on other boards.
+Driver support depends on the device you are using shimboot on. The `patch_rootfs.sh` script attempts to copy all the firmware and drivers from the shim and recovery image into the rootfs, so expect most things to work on other boards. ARM Chromebooks are not supported at the moment.
 
-### What Works:
+### Device Compatibility Table:
-- Booting Chrome OS
+| Feature \ Board Name | [`dedede`](https://chrome100.dev/board/dedede/) | [`octopus`](https://chrome100.dev/board/octopus/) | [`nissa`](https://chrome100.dev/board/nissa/) | [`reks`](https://chrome100.dev/board/reks/) | [`kefka`](https://chrome100.dev/board/kefka) | [`zork`](https://chrome100.dev/board/zork) | [`grunt`](https://chrome100.dev/board/grunt) |
-- Systemd
+|----------------------|-------------------------------------------------|---------------------------------------------------|-----------------------------------------------|---------------------------------------------|----------------------------------------------|--------------------------------------------|----------------------------------------------|
-- X11
+| X11                  | yes                                             | yes                                               | yes                                           | no <sup>[1]</sup>                           | no <sup>[1]</sup>                            | yes                                        | yes                                          |
-- XFCE
+| Wifi                 | yes                                             | yes                                               | yes                                           | yes                                         | yes                                          | yes                                        | yes                                          |
-- Backlight
+| Internal Audio       | no                                              | yes                                               | no                                            | untested                                    | yes                                          | no                                         | no                                           |
-- Touchscreen
+| Backlight            | yes                                             | yes                                               | yes                                           | untested                                    | yes                                          | untested                                   | yes                                          |
-- 3D acceleration
+| Touchscreen          | yes                                             | yes                                               | yes                                           | untested                                    | untested                                     | yes                                        | yes                                          |
-- Bluetooth
+| 3D Acceleration      | yes                                             | yes                                               | yes                                           | no                                          | no                                           | yes                                        | yes                                          |
-- Zram
+| Bluetooth            | yes                                             | yes                                               | yes                                           | untested                                    | untested                                     | yes                                        | yes                                          |
-- Wifi
+| Webcam               | yes                                             | yes                                               | yes                                           | untested                                    | untested                                     | yes                                        | yes                                          |
-- Booting a squashfs
-- Webcam
 
-### What Doesn't Work:
+<sup>1. The kernel is too old.</sup>
-- Audio (due to a firmware bug on `dedede`, this works just fine on `octopus`)
+
+This table is incomplete. If you want to contribute a device compatibility report please create a new issue on the Github repository.
+
+On all devices, expect the following features to work:
+- Zram (compressed memory)
+- Disk compression with squashfs
+
+On all devices, the following features will not work:
 - Suspend (disabled by the kernel)
 - Swap (disabled by the kernel)
 
-### Development Roadmap:
+### TODO:
-- ~~build the image automatically~~
+- Finish Python TUI rewrite (see the `python` branch if you want to help with this)
-- ~~boot to a shell~~
-- ~~switch_root into an actual rootfs~~
-- ~~start X11 in the actual rootfs~~
-- ~~ui improvements in the bootloader~~
-- ~~load all needed drivers~~
-- ~~autostart X11~~
-- ~~host repo for patched systemd packages~~
-- ~~use debootstrap to install debian~~
-- ~~prompt user for hostname and account when creating the rootfs~~
-- ~~auto load iwlmvm~~
-- ~~get wifi fully working~~
-- ~~host prebuilt images~~
-- ~~write detailed documentation~~
-- Finish Python TUI rewrite
-
-### Long Term Goals:
 - Transparent disk compression
 - Full disk encryption
-- eliminate binwalk dependency
+- Support for more distros (Ubuntu and Arch maybe)
-- get audio to work
+- Support for ARM based Chromebooks (see [issue #8](https://github.com/ading2210/shimboot/issues/8))
-- get kexec working
+- Eliminate binwalk dependency
+- Get audio to work on dedede
+- Get kexec working
+
+PRs and contributions are welcome to help implement these features.
 
 ## Usage:
 
 ### Prerequisites:
 - A separate Linux PC for the build process (preferably something Debian-based)
-- A USB that is at least 8GB in size
+  - WSL2 is supported if you are on Windows
+  - Github Codespaces is not supported at the moment
+- A USB drive that is at least 8GB in size
 - At least 20GB of free disk space
 - An x86-based Chromebook
 
@@ -87,26 +93,32 @@ Alternatively, you can run each of the steps manually:
 6. Run `sudo ./patch_rootfs.sh path_to_shim path_to_reco data/rootfs` to patch the base rootfs and add any needed drivers.
 7. Run `sudo ./build.sh image.bin path_to_shim data/rootfs` to generate a disk image at `image.bin`. 
 
+[Prebuilt images](https://github.com/ading2210/shimboot/releases) are available if you don't have a suitable device to run the build on.
+
 ### Booting the Image:
-1. Obtain a shimboot image by downloading a [prebuilt one](https://github.com/ading2210/shimboot/actions?query=branch%3Amain) or building it yourself. 
+1. Obtain a shimboot image by downloading a [prebuilt one](https://github.com/ading2210/shimboot/releases) or building it yourself. 
 2. Flash the shimboot image to a USB drive or SD card. Use the [Chromebook Recovery Utility](https://chrome.google.com/webstore/detail/chromebook-recovery-utili/pocpnlppkickgojjlmhdmidojbmbodfm) or [dd](https://linux.die.net/man/1/dd) if you're on Linux.
 3. Enable developer mode on your Chromebook. If the Chromebook is enrolled, follow the instructions on the [sh1mmer website](https://sh1mmer.me) (see the "Executing on Chromebook" section).
 4. Plug the USB into your Chromebook and enter recovery mode. It should detect the USB and run the shimboot bootloader.
 5. Boot into Debian and log in with the username and password that you configured earlier. The default username/password for the prebuilt images is `user/user`.
 6. Expand the rootfs partition so that it fills up the entire disk by running `sudo growpart /dev/sdX 4` (replacing `sdX` with the block device corresponding to your disk) to expand the partition, then running `sudo resize2fs /dev/sdX4` to expand the filesystem.
-7. Change the root password and regular user password by running `sudo passwd root` and `passwd user`.
+7. Change the user password by running `passwd user`. The root user is disabled by default.
 
 ## FAQ:
 
 #### I want to use a different Linux distribution. How can I do that?
-Using any Linux distro is possible, provided that you apply the [proper patches](https://github.com/ading2210/chromeos-systemd) to systemd and recompile it. Most distros have some sort of bootstrapping tool that allows you to install it to a directory on your host PC. Then, you can just pass that rootfs dir into `build.sh`.
+Using any Linux distro is possible, provided that you apply the [proper patches](https://github.com/ading2210/chromeos-systemd) to systemd and recompile it. Most distros have some sort of bootstrapping tool that allows you to install it to a directory on your host PC. Then, you can just pass that rootfs directory into `patch_rootfs.sh` and `build.sh`.
 
-Debian Sid (the unstable rolling release version of Debian) is also supported if you just want newer packages, and you can install it by passing an argument to `build_rootfs.sh`: 
+Debian Sid (the rolling release version of Debian) is also supported if you just want newer packages, and you can install it by passing an argument to `build_rootfs.sh`: 
 ```bash
 sudo ./build_rootfs.sh data/rootfs unstable
 ```
 #### How can I install a desktop environment other than XFCE?
-You can pass another argument to the `build_rootfs.sh` script, like this: `sudo ./build_rootfs.sh data/rootfs bookworm custom_packages=task-lxde-desktop`. The `custom_packages` argument is a list of packages (separated by spaces) that will be installed in the place of XFCE. 
+You can pass the `desktop` argument to the `build_complete.sh` script, like this:
+```bash
+sudo ./build_complete.sh grunt desktop=lxde
+```
+The valid values for this argument are: `gnome`, `xfce`, `kde`, `lxde`, `gnome-flashback`, `cinnamon`, `mate`, and `lxqt`.
 
 #### Will this prevent me from using Chrome OS normally?
 Shimboot does not touch the internal storage at all, so you will be able to use Chrome OS as if nothing happened. However, if you are on an enterprise enrolled device, booting Chrome OS again will force a powerwash due to the attempted switch into developer mode.

bootloader/bin/bootstrap.sh

@@ -28,6 +28,20 @@ enable_debug_console() {
   invoke_terminal "${tty}" "[Bootstrap Debug Console]" "/bin/busybox sh"
 }
 
+#get a partition block device from a disk path and a part number
+get_part_dev() {
+  local disk="$1"
+  local partition="$2"
+
+  #disk paths ending with a number will have a "p" before the partition number
+  last_char="$(echo -n "$disk" | tail -c 1)"
+  if [ "$last_char" -eq "$last_char" ] 2>/dev/null; then
+    echo "${disk}p${partition}"
+  else
+    echo "${disk}${partition}"
+  fi
+}
+
 find_rootfs_partitions() {
   local disks=$(fdisk -l | sed -n "s/Disk \(\/dev\/.*\):.*/\1/p")
   if [ ! "${disks}" ]; then
@@ -40,7 +54,7 @@ find_rootfs_partitions() {
       continue
     fi
     for partition in $partitions; do
-      echo "${disk}${partition}"
+      get_part_dev "$disk" "$partition"
     done
   done
 }
@@ -80,7 +94,7 @@ move_mounts() {
 
 print_license() {
   cat << EOF 
-Shimboot v1.0.0
+Shimboot v1.0.2
 
 ading2210/shimboot: Boot desktop Linux from a Chrome OS RMA shim.
 Copyright (C) 2023 ading2210
@@ -182,25 +196,6 @@ contains_word() {
   return 1
 }
 
-#might be useful in case we need to disable the tpm
-#currently this causes a kernel panic when we try to boot cros
-unbind_driver() {
-  local driver_path="$1"
-  local sys_files="$(ls $driver_path)"
-  local excluded_files="bind uevent unbind"
-  for file in $sys_files; do
-    if ! contains_word "$file" "$excluded_files"; then
-      echo "$file" > "${driver_path}/unbind"
-    fi
-  done
-}
-
-unbind_tpm() {
-  unbind_driver "/sys/bus/spi/drivers/tpm_tis_spi"
-  unbind_driver "/sys/bus/pnp/drivers/tpm_tis"
-  unbind_driver "/sys/bus/platform/drivers/tpm_tis"
-}
-
 copy_progress() {
   local source="$1"
   local destination="$2"

build_complete.sh

@@ -1,6 +1,7 @@
 #!/bin/bash
 
 . ./common.sh
+. ./image_utils.sh
 
 print_help() {
   echo "Usage: ./build_complete.sh board_name"
@@ -8,18 +9,27 @@ print_help() {
   echo "  compress_img - Compress the final disk image into a zip file. Set this to any value to enable this option."
   echo "  rootfs_dir   - Use a different rootfs for the build. The directory you select will be copied before any patches are applied."
   echo "  quiet        - Don't use progress indicators which may clog up log files."
+  echo "  desktop      - The desktop environment to install. This defaults to 'xfce'. Valid options include:"
+  echo "                   gnome, xfce, kde, lxde, gnome-flashback, cinnamon, mate, lxqt"
+  echo "  data_dir     - The working directory for the scripts. This defaults to ./data"
 }
 
 assert_root
 assert_args "$1"
 parse_args "$@"
 
-needed_deps="wget python3 unzip zip git debootstrap cpio binwalk pcregrep cgpt mkfs.ext4 mkfs.ext2 fdisk rsync"
+compress_img="${args['compress_img']}"
+rootfs_dir="${args['rootfs_dir']}"
+quiet="${args['quiet']}"
+desktop="${args['desktop']-'xfce'}"
+data_dir="${args['data_dir']}"
+
+needed_deps="wget python3 unzip zip git debootstrap cpio binwalk pcregrep cgpt mkfs.ext4 mkfs.ext2 fdisk rsync depmod findmnt"
 if [ "$(check_deps "$needed_deps")" ]; then
   #install deps automatically on debian and ubuntu
   if [ -f "/etc/debian_version" ]; then
     echo "attempting to install build deps"
-    apt-get install wget python3-all unzip zip debootstrap cpio binwalk pcregrep cgpt rsync pv -y
+    apt-get install wget python3-all unzip zip debootstrap cpio binwalk pcregrep cgpt rsync kmod pv -y
   fi
   assert_deps "$needed_deps"
 fi
@@ -38,6 +48,12 @@ board="$1"
 shim_url="https://dl.darkn.bio/api/raw/?path=/SH1mmer/$board.zip"
 boards_url="https://chromiumdash.appspot.com/cros/fetch_serving_builds?deviceCategory=ChromeOS"
 
+if [ -z "$data_dir" ]; then
+  data_dir="$base_dir/data"
+else
+  data_dir="$(realpath -m "$data_dir")"
+fi
+
 echo "downloading list of recovery images"
 reco_url="$(wget -qO- --show-progress $boards_url | python3 -c '
 import json, sys
@@ -45,25 +61,28 @@ import json, sys
 all_builds = json.load(sys.stdin)
 board = all_builds["builds"][sys.argv[1]]
 if "models" in board:
-  board = next(iter(board["models"].values()))
+  for device in board["models"].values():
+    if device["pushRecoveries"]:
+      board = device
+      break
 
 reco_url = list(board["pushRecoveries"].values())[-1]
 print(reco_url)
 ' $board)"
 echo "found url: $reco_url"
 
-shim_bin="$base_dir/data/shim_$board.bin"
+shim_bin="$data_dir/shim_$board.bin"
-shim_zip="$base_dir/data/shim_$board.zip"
+shim_zip="$data_dir/shim_$board.zip"
-reco_bin="$base_dir/data/reco_$board.bin"
+reco_bin="$data_dir/reco_$board.bin"
-reco_zip="$base_dir/data/reco_$board.zip"
+reco_zip="$data_dir/reco_$board.zip"
-mkdir -p "$base_dir/data"
+mkdir -p "$data_dir"
 
 download_and_unzip() {
   local url="$1"
   local zip_path="$2"
   local bin_path="$3"
   if [ ! -f "$bin_path" ]; then
-    if [ ! "${args['quiet']}" ]; then
+    if [ ! "$quiet" ]; then
       wget -q --show-progress $url -O $zip_path -c
     else
       wget -q $url -O $zip_path -c
@@ -74,7 +93,7 @@ download_and_unzip() {
     cleanup_path="$bin_path"
     echo "extracting $zip_path"
     local total_bytes="$(unzip -lq $zip_path | tail -1 | xargs | cut -d' ' -f1)"
-    if [ ! "${args['quiet']}" ]; then
+    if [ ! "$quiet" ]; then
       unzip -p $zip_path | pv -s $total_bytes > $bin_path
     else
       unzip -p $zip_path > $bin_path
@@ -97,32 +116,37 @@ download_and_unzip $reco_url $reco_zip $reco_bin
 echo "downloading shim image"
 download_and_unzip $shim_url $shim_zip $shim_bin
 
-if [ ! "${args['rootfs_dir']}" ]; then
+if [ ! "$rootfs_dir" ]; then
   rootfs_dir="$(realpath -m data/rootfs_$board)"
+  desktop_package="task-$desktop-desktop"
+  if [ "$(findmnt -T "$rootfs_dir/dev")" ]; then
+    sudo umount -l $rootfs_dir/* 2>/dev/null || true
+  fi
   rm -rf $rootfs_dir
   mkdir -p $rootfs_dir
 
   echo "building debian rootfs"
   ./build_rootfs.sh $rootfs_dir bookworm \
+    custom_packages=$desktop_package \
     hostname=shimboot-$board \
-    root_passwd=root \
     username=user \
     user_passwd=user
-else
-  rootfs_dir="$(realpath -m "${args['rootfs_dir']}")"
 fi
 
 echo "patching debian rootfs"
-retry_cmd ./patch_rootfs.sh $shim_bin $reco_bin $rootfs_dir "quiet=${args['quiet']}"
+retry_cmd ./patch_rootfs.sh $shim_bin $reco_bin $rootfs_dir "quiet=$quiet"
 
 echo "building final disk image"
-final_image="$base_dir/data/shimboot_$board.bin"
+final_image="$data_dir/shimboot_$board.bin"
 rm -rf $final_image
-retry_cmd ./build.sh $final_image $shim_bin $rootfs_dir "quiet=${args['quiet']}"
+retry_cmd ./build.sh $final_image $shim_bin $rootfs_dir "quiet=$quiet"
 echo "build complete! the final disk image is located at $final_image"
 
-if [ "${args['compress_img']}" ]; then
+echo "cleaning up"
-  image_zip="$base_dir/data/shimboot_$board.zip"
+clean_loops
+
+if [ "$compress_img" ]; then
+  image_zip="$data_dir/shimboot_$board.zip"
   echo "compressing disk image into a zip file"
   zip -j $image_zip $final_image
   echo "finished compressing the disk file"

build_rootfs.sh

@@ -14,14 +14,16 @@ print_help() {
   echo "Valid named arguments (specify with 'key=value'):"
   echo "  custom_packages - The packages that will be installed in place of task-xfce-desktop."
   echo "  hostname        - The hostname for the new rootfs."
-  echo "  root_passwd     - The root password."
+  echo "  enable_root     - Enable the root user."
+  echo "  root_passwd     - The root password. This only has an effect if enable_root is set."
   echo "  username        - The unprivileged user name for the new rootfs."
   echo "  user_passwd     - The password for the unprivileged user."
+  echo "  disable_base    - Disable the base packages such as zram, cloud-utils, and command-not-found."
   echo "If you do not specify the hostname and credentials, you will be prompted for them later."
 }
 
 assert_root
-assert_deps "realpath debootstrap"
+assert_deps "realpath debootstrap findmnt"
 assert_args "$2"
 parse_args "$@"
 
@@ -38,6 +40,22 @@ unmount_all() {
   done
 }
 
+need_remount() {
+  local target="$1"
+  local mnt_options="$(findmnt -T "$target" | tail -n1 | rev | cut -f1 -d' '| rev)"
+  echo "$mnt_options" | grep -e "noexec" -e "nodev"
+}
+
+do_remount() {
+  local target="$1"
+  local mountpoint="$(findmnt -T "$target" | tail -n1 | cut -f1 -d' ')"
+  mount -o remount,dev,exec "$mountpoint"
+}
+
+if [ "$(need_remount "$rootfs_dir")" ]; then
+  do_remount "$rootfs_dir"
+fi
+
 debootstrap --arch amd64 $release_name $rootfs_dir http://deb.debian.org/debian/
 cp -ar rootfs/* $rootfs_dir
 cp /etc/resolv.conf $rootfs_dir/etc/resolv.conf
@@ -49,11 +67,18 @@ done
 
 hostname="${args['hostname']}"
 root_passwd="${args['root_passwd']}"
+enable_root="${args['enable_root']}"
 username="${args['username']}"
 user_passwd="${args['user_passwd']}"
+disable_base="${args['disable_base']}"
+
+chroot_command="/opt/setup_rootfs.sh \
+  '$DEBUG' '$release_name' '$packages' \
+  '$hostname' '$root_passwd' '$username' \
+  '$user_passwd' '$enable_root' '$disable_base'"
+
+LC_ALL=C chroot $rootfs_dir /bin/bash -c "${chroot_command}"
 
-chroot_command="/opt/setup_rootfs.sh '$DEBUG' '$release_name' '$packages' '$hostname' '$root_passwd' '$username' '$user_passwd'"
-chroot $rootfs_dir /bin/bash -c "${chroot_command}"
 trap - EXIT
 unmount_all
 

common.sh

@@ -29,6 +29,11 @@ assert_deps() {
 parse_args() {
   declare -g -A args
   for argument in "$@"; do
+    if [ "$argument" = "-h" ] || [ "$argument" = "--help" ]; then
+      print_help
+      exit 0
+    fi
+
     local key=$(echo $argument | cut -f1 -d=)
     local key_length=${#key}
     local value="${argument:$key_length+1}"

image_utils.sh

@@ -2,15 +2,16 @@
 
 create_loop() {
   local loop_device=$(losetup -f)
+  if [ ! -b "$loop_device" ]; then
+    #we might run out of loop devices, see https://stackoverflow.com/a/66020349
+    local major=$(grep loop /proc/devices | cut -c3)
+    local number="$(echo "$loop_device" | grep -Eo '[0-9]+' | tail -n1)"
+    mknod $loop_device b $major $number
+  fi
   losetup -P $loop_device "${1}"
   echo $loop_device
 }
 
-#original shim rootfses have a non standard ext2 filesystem
-make_mountable() {
-  printf '\000' | dd of=$1 seek=$((0x464 + 3)) conv=notrunc count=1 bs=1 status=none
-}
-
 #set required flags on the kernel partition
 make_bootable() {
   cgpt add -i 2 -S 1 -T 5 -P 10 -l kernel $1
@@ -65,10 +66,18 @@ partition_disk() {
 }
 
 safe_mount() {
-  umount $2 2> /dev/null || /bin/true
+  local source="$1"
-  rm -rf $2
+  local dest="$2"
-  mkdir -p $2
+  local opts="$3"
-  mount $1 $2
+  
+  umount $dest 2> /dev/null || /bin/true
+  rm -rf $dest
+  mkdir -p $dest
+  if [ "$opts" ]; then
+    mount $source $dest -o $opts
+  else
+    mount $source $dest
+  fi
 }
 
 create_partitions() {
@@ -138,3 +147,16 @@ patch_initramfs() {
 
   find ${initramfs_path}/bin -name "*" -exec chmod +x {} \;
 }
+
+#clean up unused loop devices
+clean_loops() {
+  local loop_devices="$(losetup -a | awk -F':' {'print $1'})"
+  for loop_device in $loop_devices; do
+    local mountpoints="$(cat /proc/mounts | grep "$loop_device")"
+    if [ ! "$mountpoints" ]; then
+      losetup -d $loop_device
+    else
+      echo "warning: not removing $loop_device because it is still mounted"
+    fi
+  done
+}

patch_rootfs.sh

@@ -2,26 +2,16 @@
 
 #patch the target rootfs to add any needed drivers
 
-set -e
+. ./common.sh
-if [ "$DEBUG" ]; then
-  set -x
-fi
-
 . ./image_utils.sh
 
 print_help() {
   echo "Usage: ./patch_rootfs.sh shim_path reco_path rootfs_dir"
 }
 
-if [ "$EUID" -ne 0 ]; then
+assert_root
-  echo "this needs to be run as root."
+assert_deps "git gunzip depmod"
-  exit 1
+assert_args "$3"
-fi
-
-if [ -z "$3" ]; then
-  print_help
-  exit 1
-fi
 
 copy_modules() {
   local shim_rootfs=$(realpath -m $1)
@@ -39,6 +29,16 @@ copy_modules() {
   mkdir -p "${target_rootfs}/etc/modprobe.d/"
   cp -r "${reco_rootfs}/lib/modprobe.d/"* "${target_rootfs}/lib/modprobe.d/"
   cp -r "${reco_rootfs}/etc/modprobe.d/"* "${target_rootfs}/etc/modprobe.d/"
+
+  #decompress kernel modules if necessary - debian won't recognize these otherwise
+  local compressed_files="$(find "${target_rootfs}/lib/modules" -name '*.gz')"
+  if [ "$compressed_files" ]; then
+    echo "$compressed_files" | xargs gunzip
+    for kernel_dir in "$target_rootfs/lib/modules/"*; do
+      local version="$(basename "$kernel_dir")"
+      depmod -b "$target_rootfs" "$version"
+    done
+  fi
 }
 
 copy_firmware() {
@@ -67,13 +67,11 @@ reco_rootfs="/tmp/reco_rootfs"
 
 echo "mounting shim"
 shim_loop=$(create_loop "${shim_path}")
-make_mountable "${shim_loop}p3"
+safe_mount "${shim_loop}p3" $shim_rootfs ro
-safe_mount "${shim_loop}p3" $shim_rootfs
 
 echo "mounting recovery image"
 reco_loop=$(create_loop "${reco_path}")
-make_mountable "${reco_loop}p3"
+safe_mount "${reco_loop}p3" $reco_rootfs ro
-safe_mount "${reco_loop}p3" $reco_rootfs
 
 echo "copying modules to rootfs"
 copy_modules $shim_rootfs $reco_rootfs $target_rootfs

rootfs/opt/setup_rootfs.sh

@@ -16,12 +16,14 @@ hostname="$4"
 root_passwd="$5"
 username="$6"
 user_passwd="$7"
+enable_root="$8"
+disable_base_pkgs="$9"
 
 custom_repo="https://shimboot.ading.dev/debian"
 custom_repo_domain="shimboot.ading.dev"
 sources_entry="deb [trusted=yes arch=amd64] ${custom_repo} ${release_name} main"
 
-export DEBIAN_FRONTEND=noninteractive
+export DEBIAN_FRONTEND="noninteractive"
 
 #add shimboot repos
 echo -e "${sources_entry}\n$(cat /etc/apt/sources.list)" > /etc/apt/sources.list
@@ -40,14 +42,17 @@ apt-get install --reinstall $installed_systemd
 #enable shimboot services
 systemctl enable kill-frecon.service
 
-#install desktop
+#install base packages
-apt-get install -y $packages cloud-utils zram-tools
+if [ -z "$disable_base_pkgs" ]; then
+  apt-get install -y cloud-utils zram-tools sudo command-not-found bash-completion
 
-#set up zram
+  #set up zram
-tee -a /etc/default/zramswap << END
+  echo "ALGO=lzo" >> /etc/default/zramswap
-ALGO=lzo
+  echo "PERCENT=50" >> /etc/default/zramswap
-PERCENT=50
+
-END
+  #update apt-file cache
+  apt-file update
+fi
 
 #set up hostname and username
 if [ ! "$hostname" ]; then
@@ -64,28 +69,35 @@ ff02::1 ip6-allnodes
 ff02::2 ip6-allrouters
 END
 
-echo "Enter a root password:"
+#install desktop and other custom packages
-if [ ! "$root_passwd" ]; then
+apt-get install -y $packages
-  while ! passwd root; do
-    echo "Failed to set password, please try again."
-  done
-else
-  yes "$root_passwd" | passwd root
-fi
 
 if [ ! $username ]; then
   read -p "Enter the username for the user account: " username
 fi
 useradd -m -s /bin/bash -G sudo $username
 
-if [ ! "$user_passwd" ]; then
+set_password() {
-  echo "Enter the password for ${username}:"
+  local user="$1"
-  while ! passwd $username; do
+  local password="$2"
-    echo "Failed to set password, please try again."
+  if [ ! "$password" ]; then
+    while ! passwd $user; do
+      echo "Failed to set password for $user, please try again."
     done
+  else
+    yes "$password" | passwd $user
+  fi
+}
+
+if [ "$enable_root" ]; then 
+  echo "Enter a root password:"
+  set_password root "$root_passwd"
 else
-  yes "$user_passwd" | passwd $username
+  usermod -a -G sudo $username
 fi
 
+echo "Enter a user password:"
+set_password "$username" "$user_passwd"
+
 #clean apt caches
 apt-get clean

website/index.html

@@ -68,7 +68,7 @@ Another problem is encountered at this stage: the Chrome OS kernel will complain
 After copying all the firmware from the recovery image and shim to the rootfs, we're able to boot to a mostly working XFCE desktop.
 
 <b id="prebuilt">Prebuilt Images:</b>
-Prebuilt images are available <a href="https://github.com/ading2210/shimboot/actions">here</a>. If there is not a prebuilt image for your board, you must manually build the shimboot image. 
+Prebuilt images are available <a href="https://github.com/ading2210/shimboot/releases">here</a>. If there is not a prebuilt image for your board, you must manually build the shimboot image. 
 
 For these images, the root password is "root". The name of the default user is "user" and its password is "user" as well. You should change these credentials as soon as possible.
 
@@ -86,12 +86,13 @@ Alternatively, you can run each of the steps manually:
 6. Run `sudo ./build.sh image.bin path_to_shim data/rootfs` to generate a disk image at `image.bin`. 
 
 <b id="usage">Booting the Image:</b>
-1. Obtain a shimboot image by downloading a <a href="https://github.com/ading2210/shimboot/actions?query=branch%3Amain">prebuilt one</a> or building it yourself. 
+1. Obtain a shimboot image by downloading a <a href="https://github.com/ading2210/shimboot/releases">prebuilt one</a> or building it yourself. 
 2. Flash the shimboot image to a USB drive or SD card. Use the <a href="https://chrome.google.com/webstore/detail/chromebook-recovery-utili/pocpnlppkickgojjlmhdmidojbmbodfm">Chromebook Recovery Utility</a> or <a href="https://linux.die.net/man/1/dd">dd</a> if you're on Linux.
 3. Enable developer mode on your Chromebook. If the Chromebook is enrolled, follow the instructions on the <a href="https://sh1mmer.me">sh1mmer website</a> (see the "Executing on Chromebook" section).
 4. Plug the USB into your Chromebook and enter recovery mode. It should detect the USB and run the shimboot bootloader.
 5. Boot into Debian and log in with the username and password that you configured earlier. The default username/password for the prebuilt images is `user/user`.
 6. Expand the rootfs partition so that it fills up the entire disk by running `sudo growpart /dev/sdX 4` (replacing `sdX` with the block device corresponding to your disk) to expand the partition, then running `sudo resize2fs /dev/sdX4` to expand the filesystem.
+7. Change the root password and regular user password by running `sudo passwd root` and `passwd user`.
 
 <b id="copyright">Copyright:</b>
 Shimboot is licensed under the <a href="https://www.gnu.org/licenses/gpl-3.0.txt">GNU GPL v3</a>. Unless otherwise indicated, all code has been written by me, <a href="https://ading.dev">ading2210</a>.