armbian-build/extensions/flash-kernel.sh

123 lines
7.7 KiB
Bash
Raw Normal View History

extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
# This runs *after* user_config. Don't change anything not coming from other variables or meant to be configured by the user.
function extension_prepare_config__prepare_flash_kernel() {
# Configuration defaults, or lack thereof.
export FK__TOOL_PACKAGE="${FK__TOOL_PACKAGE:-flash-kernel}"
export FK__PUBLISHED_KERNEL_VERSION="${FK__PUBLISHED_KERNEL_VERSION:-undefined-flash-kernel-version}"
export FK__EXTRA_PACKAGES="${FK__EXTRA_PACKAGES:-undefined-flash-kernel-kernel-package}"
export FK__KERNEL_PACKAGES="${FK__KERNEL_PACKAGES:-}"
export FK__MACHINE_MODEL="${FK__MACHINE_MODEL:-Undefined Flash-Kernel Machine}"
# Override certain variables. A case of "this extension knows better and modifies user configurable stuff".
export BOOTCONFIG="none" # To try and convince lib/ to not build or install u-boot.
unset BOOTSOURCE # To try and convince lib/ to not build or install u-boot.
export UEFISIZE=256 # in MiB. Not really UEFI, but partition layout is the same.
export BOOTSIZE=0 # No separate /boot, flash-kernel will "flash" the kernel+initrd to the firmware part.
export UEFI_MOUNT_POINT="/boot/firmware" # mount uefi partition at /boot/firmware
export CLOUD_INIT_CONFIG_LOCATION="/boot/firmware" # use /boot/firmware for cloud-init as well
export VER="${FK__PUBLISHED_KERNEL_VERSION}" # For the VERSION
export EXTRA_BSP_NAME="${EXTRA_BSP_NAME}-fk${FK__PUBLISHED_KERNEL_VERSION}" # Unique bsp name.
2022-10-08 11:17:42 +00:00
echo "-- starting" > "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log # Zero out the log for this extension.
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
}
function post_install_kernel_debs__install_kernel_and_flash_packages() {
export INSTALL_ARMBIAN_FIRMWARE="no" # Disable Armbian-firmware install, which would happen after this method.
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
if [[ "${FK__EXTRA_PACKAGES}" != "" ]]; then
display_alert "Installing flash-kernel extra packages" "${FK__EXTRA_PACKAGES}"
2022-10-08 11:17:42 +00:00
echo "-- install extra pkgs" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log
chroot "${SDCARD}" /bin/bash -c "DEBIAN_FRONTEND=noninteractive apt-get ${APT_EXTRA_DIST_PARAMS} -yqq --no-install-recommends install ${FK__EXTRA_PACKAGES}" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log || {
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
display_alert "Failed to install flash-kernel's extra packages." "${EXTENSION}" "err"
exit 28
}
fi
if [[ "${FK__KERNEL_PACKAGES}" != "" ]]; then
display_alert "Installing flash-kernel kernel packages" "${FK__KERNEL_PACKAGES}"
2022-10-08 11:17:42 +00:00
echo "-- install kernel pkgs" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log
chroot "${SDCARD}" /bin/bash -c "DEBIAN_FRONTEND=noninteractive apt-get ${APT_EXTRA_DIST_PARAMS} -yqq --no-install-recommends install ${FK__KERNEL_PACKAGES}" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log || {
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
display_alert "Failed to install flash-kernel's kernel packages." "${EXTENSION}" "err"
exit 28
}
fi
display_alert "Installing flash-kernel package" "${FK__TOOL_PACKAGE}"
# Create a fake /sys/firmware/efi directory so that flash-kernel does not try to do anything when installed
# @TODO: this might or not work after flash-kernel 3.104 or later
umount "${SDCARD}"/sys
mkdir -p "${SDCARD}"/sys/firmware/efi
2022-10-08 11:17:42 +00:00
echo "-- install flash-kernel package" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log
chroot "${SDCARD}" /bin/bash -c "DEBIAN_FRONTEND=noninteractive apt-get ${APT_EXTRA_DIST_PARAMS} -yqq --no-install-recommends install ${FK__TOOL_PACKAGE}" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log || {
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
display_alert "Failed to install flash-kernel package." "${EXTENSION}" "err"
exit 28
}
# Remove fake /sys/firmware (/efi) directory
rm -rf "${SDCARD}"/sys/firmware
}
# @TODO: extract u-boot into an extension, so that core bsps don't have this stuff in there to begin with.
# @TODO: this code is duplicated in grub.sh extension, so another reason to refactor the root of the evil
post_family_tweaks_bsp__remove_uboot_flash_kernel() {
display_alert "Removing uboot from BSP" "${EXTENSION}" "info"
# Simply remove everything with 'uboot' or 'u-boot' in their filenames from the BSP package.
# shellcheck disable=SC2154 # $destination is the target dir of the bsp building function
find "$destination" -type f | grep -e "uboot" -e "u-boot" | xargs rm
}
pre_umount_final_image__remove_uboot_initramfs_hook_flash_kernel() {
# even if BSP still contained this (cached .deb), make sure by removing from ${MOUNT}
[[ -f "$MOUNT"/etc/initramfs/post-update.d/99-uboot ]] && rm -v "$MOUNT"/etc/initramfs/post-update.d/99-uboot
}
function pre_update_initramfs__setup_flash_kernel() {
local chroot_target=$MOUNT
cp /usr/bin/"$QEMU_BINARY" "$chroot_target"/usr/bin/
mount_chroot "$chroot_target/" # this already handles /boot/firmware which is required for it to work.
# hack, umount the chroot's /sys, otherwise flash-kernel tries to EFI flash due to the build host (!) being EFI
umount "$chroot_target/sys"
2022-10-08 11:17:42 +00:00
echo "-- flash-kernel disabling hooks" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log
chroot "$chroot_target" /bin/bash -c "chmod -v -x /etc/kernel/postinst.d/initramfs-tools" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log 2>&1
chroot "$chroot_target" /bin/bash -c "chmod -v -x /etc/initramfs/post-update.d/flash-kernel" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log 2>&1
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
export FIRMWARE_DIR="${MOUNT}"/boot/firmware
2022-10-08 11:17:42 +00:00
call_extension_method "pre_initramfs_flash_kernel" <<- 'PRE_INITRAMFS_FLASH_KERNEL'
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
*prepare to update-initramfs before flashing kernel via flash_kernel*
A good spot to write firmware config to ${FIRMWARE_DIR} (/boot/firmware) before flash-kernel actually runs.
PRE_INITRAMFS_FLASH_KERNEL
local update_initramfs_cmd="update-initramfs -c -k all"
display_alert "Updating flash-kernel initramfs..." "$update_initramfs_cmd" ""
2022-10-08 11:17:42 +00:00
echo "-- flash-kernel initramfs" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log
chroot "$chroot_target" /bin/bash -c "$update_initramfs_cmd" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log 2>&1 || {
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
display_alert "Failed to run '$update_initramfs_cmd'" "Check ${DEST}/"${LOG_SUBPATH}"/flash-kernel.log" "err"
exit 29
}
2022-10-08 11:17:42 +00:00
call_extension_method "pre_flash_kernel" <<- 'PRE_FLASH_KERNEL'
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
*run before running flash-kernel*
Each board might need different stuff for flash-kernel to work. Implement it here.
Write to `${MOUNT}`, eg: `"${MOUNT}"/etc/flash-kernel`
PRE_FLASH_KERNEL
local flash_kernel_cmd="FK_FORCE=yes flash-kernel --machine '${FK__MACHINE_MODEL}'" # FK_FORCE=yes is required since flash-kernel 3.104ubuntu14 / 3.106ubuntu7
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
display_alert "flash-kernel" "${FK__MACHINE_MODEL}" "info"
2022-10-08 11:17:42 +00:00
echo "-- flash-kernel itself" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log
chroot "$chroot_target" /bin/bash -c "${flash_kernel_cmd}" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log 2>&1 || {
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
display_alert "Failed to run '${flash_kernel_cmd}'" "Check ${DEST}/"${LOG_SUBPATH}"/flash-kernel.log" "err"
exit 29
}
display_alert "Re-enabling" "initramfs-tools/flash-kernel hook for kernel"
2022-10-08 11:17:42 +00:00
echo "-- flash-kernel re-enabling hooks" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log
chroot "$chroot_target" /bin/bash -c "chmod -v +x /etc/kernel/postinst.d/initramfs-tools" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log 2>&1
chroot "$chroot_target" /bin/bash -c "chmod -v +x /etc/initramfs/post-update.d/flash-kernel" >> "${DEST}"/"${LOG_SUBPATH}"/flash-kernel.log 2>&1
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
umount_chroot "$chroot_target/"
rm "$chroot_target"/usr/bin/"$QEMU_BINARY"
display_alert "Disabling Armbian-core update_initramfs, was already done above." "${EXTENSION}"
unset KERNELSOURCE # ugly. sorry. we'll have better mechanism for this soon. this is tested at lib/debootstrap.sh:844
extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300) * extensions framework (née "fragments") - this should actually change nothing at this point, just add capabilities - the framework is implemented in lib/extensions.sh - the "if function x exists then call x" replaced with call_extension_method() - +inline documentation - +compatibility names Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; meta-extensions: auto-docs and sample extension gen - 2 extensions dealing with extensibility itself - detect-unused-extensions: shows which extensions are enabled, but never called. - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * new extension methods and features via config variables in core Armbian - `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones - `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file - `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP - `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images - `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs" - `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al - `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware - `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64 - `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image - define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al - initial support for targeting x86/amd64 UEFI and BIOS - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends up at `$uefipart`=15) - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14) - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac - hard-requires gdisk package host-side - add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains download - add pre_prepare_partitions() extension method, for custom partition size calculations - add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes` - add post_create_partitions() extension method, mostly for easy debugging - add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification - add post_install_kernel_debs() extension method. - multiple fixes to bsp to avoid spurious errors when files are not where it expects - v4: detect `update-initramfs` failure and abort build with useful message if it does - v4: show useful stacktrace in `exit_with_error` - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong - v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand - v4: preserve kernel .config's dates when copying Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions framework; refactor tool fetching/building into extensions - a few examples of core refactoring using extensions - sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example) - marvel-tools extension, enabled by 2 different mvebu family includes - rkbin-tools extension, enabled by rockship64_common family include - amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example) - removed the 'testings' fetch_from_repo completely since not used anywhere. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension - v3: added `growroot`-awareness to `armbian-resize-filesystem` - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs - now `armbian-resize-filesystem` handles `resize2fs` only, and works. - v4: reworked UEFI board/family/include structure: - use Distro's `linux-generic` kernel only for `current` - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64 - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al - v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb` - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it - v5: many fixes - v7: more small fixes. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * .wip for the RaspberryPi 4B via new flash-kernel extension - this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu - flash-kernel is not really a bootloader - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases - it is really only known-working since Hirsute release. - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels. - v3: fixed focal rootfs build. untested. - v3: better variable names, preparing for source-built kernel. - v5: new edge build with pure mainline kernel. - v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's) Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Added first patch to edge x86 related to wifi drivers * extensions: leave hostapd alone; remove hackish ext; block reentrancy - package-list-utils does not belong in this PR - grub or bcm2711 is not the place to remove hostapd - block recursive enable_extension() calls, for now. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * gen-sample-extension-docs: fix: avoid counter in generated sample Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: dependencies: enable_extension() in extensions with a stack - and better stacktraces, I hope Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Remove code from package list since we don't have it in repository Adjust kernel config to disable driver that needs further polishing. * Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work * amd64: allow building amd64 on aarch64 with system toolchain - conditionally add gcc-x86-64-linux-gnu to hostdeps Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * add libelf-dev directly to hostdeps (and Dockerfile), remove extension Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause - which was the missing $image_name for non-arm64 & non-arm, so: x86 for example Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * flash-kernel: really obliterate u-boot stuff from BSP - for now. soon we'll refactor u-boot so not have to do this Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: add host_dependencies_ready() hook - this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add verification functions for correct selection. * If UEFI Skip symlink creation * Do not create dtb package for amd64 * Skip scripts folder cleaning if build process native. Skip creating postinst prerm scripts for headers. * Skip applying headers-debian-byteshift.patch if build native * Fix architecture syntax as x86_64 * Revert "amd64: allow building amd64 on aarch64 with system toolchain" This reverts commit 0c5ee20bb1b33a133e6e359476082d43d5ad457c. * Compare architectures before starting compilation. Signed-off-by: The-going <48602507+The-going@users.noreply.github.com> * extensions: cleanups after fixes by the-Going - packaging: - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks! - Explicitly disallow "reverse cross compile" in amd64.conf. - whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today. - fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * grub: remove debug Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * firmware: allow installing `armbian-firmware-full`; make it really full - can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI. - don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes` - rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts. Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/ Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all() - to reset/unset everything done by the the initializer, so build can run again Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: remove 'global' logging, for use with build_all_ng - enable_extensions() will have to live on without logging to file. it's just too early. - now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager() - now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager() Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * extensions: build-all-ng.sh bugfix due to extension's debug to stdout - extensions (among other things) can produce output to stdout when activated - fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger - debugging output got mixed with "idential", rendering hash cache void for families that used extensions - eg: sunxi, others - fix is to send stdout to the bitbucket when sourcing the board & arch config files - proper fix would be stop using stdout in this case and use return code for check_hash() - one day soon Signed-off-by: Ricardo Pardini <ricardo@pardini.net> * Add CI build targets Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com> Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 08:49:49 +00:00
}