From patchwork Tue Mar 19 16:58:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: ar71xx: Add caldata extraction speedup From: Adrian Schmutzler X-Patchwork-Id: 1034 Message-Id: <20190319165838.3372-1-freifunk@adrianschmutzler.de> To: franken-dev@freifunk.net Date: Tue, 19 Mar 2019 17:58:38 +0100 This is a backport from the OpenWrt master: Reading and writing to and from flash storage is slowed down enormously by some functions which use a block size of 1. This patch reworks the extraction scripts to be much faster and efficient by reading and writing in possibly one big block. Speed comparison @ TP-Link TL-WDR4300 (just manually) results in a time reduction by three orders of magnitude (99.9 %). > time dd if=/dev/mtd3 of=/lib/firmware/test-slow bs=1 count=4096 skip=4096 4096+0 records in 4096+0 records out real 0m 15.85s user 0m 0.06s sys 0m 13.28s > time dd if=/dev/mtd3 of=/lib/firmware/test-fast bs=4096 count=1 skip=4096 iflag=skip_bytes 1+0 records in 1+0 records out real 0m 0.02s user 0m 0.00s sys 0m 0.02s Since the boot time increase is relatively substantial, I decided to backport this to the current, openwrt-18.06-based firmware. Signed-off-by: Adrian Schmutzler --- Just in case someone is interested in reviewing this for our firmware as long as it stays 18.06 ... --- ...1-ar71xx-Speed-up-caldata-eeprom-handling.patch | 102 +++++++++++++++++++++ ...71xx-Speed-up-mtd-extraction-in-ar71xx.sh.patch | 39 ++++++++ 2 files changed, 141 insertions(+) create mode 100755 build_patches/openwrt/0041-ar71xx-Speed-up-caldata-eeprom-handling.patch create mode 100755 build_patches/openwrt/0042-ar71xx-Speed-up-mtd-extraction-in-ar71xx.sh.patch diff --git a/build_patches/openwrt/0041-ar71xx-Speed-up-caldata-eeprom-handling.patch b/build_patches/openwrt/0041-ar71xx-Speed-up-caldata-eeprom-handling.patch new file mode 100755 index 00000000..363e173a --- /dev/null +++ b/build_patches/openwrt/0041-ar71xx-Speed-up-caldata-eeprom-handling.patch @@ -0,0 +1,102 @@ +From df6505e5adfaaf04231e459afac492ffadde69ec Mon Sep 17 00:00:00 2001 +From: Adrian Schmutzler +Date: Thu, 7 Mar 2019 09:40:08 +0100 +Subject: [PATCH 1/2] ar71xx: Speed up caldata/eeprom handling + +Reading and writing to and from flash storage is slowed down +enormously by some functions which use a block size of 1. + +This patch reworks the extraction scripts to be much faster and +efficient by reading and writing in possibly one big block. + +This is based on the initial commit a69e101 for ipq40xx by +Christian Lamparter . + +Speed comparison @ TP-Link TL-WDR4300 (just manually) results +in a time reduction by three orders of magnitude (99.9 %). + +> time dd if=/dev/mtd3 of=/lib/firmware/test-slow bs=1 count=4096 skip=4096 +4096+0 records in +4096+0 records out +real 0m 15.85s +user 0m 0.06s +sys 0m 13.28s + +> time dd if=/dev/mtd3 of=/lib/firmware/test-fast bs=4096 count=1 skip=4096 iflag=skip_bytes +1+0 records in +1+0 records out +real 0m 0.02s +user 0m 0.00s +sys 0m 0.02s + +Signed-off-by: Adrian Schmutzler +--- + .../linux/ar71xx/base-files/etc/hotplug.d/firmware/10-ath9k-eeprom | 6 +++--- + .../ar71xx/base-files/etc/hotplug.d/firmware/11-ath10k-caldata | 6 +++--- + 2 files changed, 6 insertions(+), 6 deletions(-) + +diff --git a/target/linux/ar71xx/base-files/etc/hotplug.d/firmware/10-ath9k-eeprom b/target/linux/ar71xx/base-files/etc/hotplug.d/firmware/10-ath9k-eeprom +index 94bce7d335..208d5f6bff 100644 +--- a/target/linux/ar71xx/base-files/etc/hotplug.d/firmware/10-ath9k-eeprom ++++ b/target/linux/ar71xx/base-files/etc/hotplug.d/firmware/10-ath9k-eeprom +@@ -20,7 +20,7 @@ ath9k_eeprom_extract() { + [ -n "$mtd" ] || \ + ath9k_eeprom_die "no mtd device found for partition $part" + +- dd if=$mtd of=/lib/firmware/$FIRMWARE bs=1 skip=$offset count=$count 2>/dev/null || \ ++ dd if=$mtd of=/lib/firmware/$FIRMWARE iflag=skip_bytes bs=$count skip=$offset count=1 2>/dev/null || \ + ath9k_eeprom_die "failed to extract from $mtd" + } + +@@ -35,7 +35,7 @@ ath9k_ubi_eeprom_extract() { + [ -n "$ubi" ] || \ + ath9k_eeprom_die "no UBI volume found for $part" + +- dd if=/dev/$ubi of=/lib/firmware/$FIRMWARE bs=1 skip=$offset count=$count 2>/dev/null || \ ++ dd if=/dev/$ubi of=/lib/firmware/$FIRMWARE iflag=skip_bytes bs=$count skip=$offset count=1 2>/dev/null || \ + ath9k_eeprom_die "failed to extract from $ubi" + } + +@@ -62,7 +62,7 @@ ath9k_patch_firmware_mac() { + + [ -z "$mac" ] && return + +- macaddr_2bin $mac | dd of=/lib/firmware/$FIRMWARE conv=notrunc bs=1 seek=2 count=6 ++ macaddr_2bin $mac | dd of=/lib/firmware/$FIRMWARE conv=notrunc oflag=seek_bytes bs=6 seek=2 count=1 + } + + board=$(board_name) +diff --git a/target/linux/ar71xx/base-files/etc/hotplug.d/firmware/11-ath10k-caldata b/target/linux/ar71xx/base-files/etc/hotplug.d/firmware/11-ath10k-caldata +index d21d4134d4..39551e41a8 100644 +--- a/target/linux/ar71xx/base-files/etc/hotplug.d/firmware/11-ath10k-caldata ++++ b/target/linux/ar71xx/base-files/etc/hotplug.d/firmware/11-ath10k-caldata +@@ -10,7 +10,7 @@ ath10kcal_from_file() { + local offset=$2 + local count=$3 + +- dd if=$source of=/lib/firmware/$FIRMWARE bs=1 skip=$offset count=$count 2>/dev/null || \ ++ dd if=$source of=/lib/firmware/$FIRMWARE iflag=skip_bytes bs=$count skip=$offset count=1 2>/dev/null || \ + ath10kcal_die "failed to extract calibration data from $source" + } + +@@ -30,7 +30,7 @@ ath10kcal_extract() { + [ "$count" = "$cal_size" ] || \ + ath10kcal_die "no calibration data found in $part" + +- dd if=$mtd of=/lib/firmware/$FIRMWARE bs=1 skip=$offset count=$count 2>/dev/null || \ ++ dd if=$mtd of=/lib/firmware/$FIRMWARE iflag=skip_bytes bs=$count skip=$offset count=1 2>/dev/null || \ + ath10kcal_die "failed to extract calibration data from $mtd" + } + +@@ -39,7 +39,7 @@ ath10kcal_patch_mac() { + + [ -z "$mac" ] && return + +- macaddr_2bin $mac | dd of=/lib/firmware/$FIRMWARE conv=notrunc bs=1 seek=6 count=6 ++ macaddr_2bin $mac | dd of=/lib/firmware/$FIRMWARE conv=notrunc oflag=seek_bytes bs=6 seek=6 count=1 + } + + [ -e /lib/firmware/$FIRMWARE ] && exit 0 +-- +2.11.0 + diff --git a/build_patches/openwrt/0042-ar71xx-Speed-up-mtd-extraction-in-ar71xx.sh.patch b/build_patches/openwrt/0042-ar71xx-Speed-up-mtd-extraction-in-ar71xx.sh.patch new file mode 100755 index 00000000..239b79dd --- /dev/null +++ b/build_patches/openwrt/0042-ar71xx-Speed-up-mtd-extraction-in-ar71xx.sh.patch @@ -0,0 +1,39 @@ +From 61fcbe7b1065a8aae983063c4f7002b5a562a494 Mon Sep 17 00:00:00 2001 +From: Adrian Schmutzler +Date: Thu, 7 Mar 2019 09:40:09 +0100 +Subject: [PATCH 2/2] ar71xx: Speed up mtd extraction in ar71xx.sh + +Although the amount of data read here is smaller than for the +caldata, there still might be some speed gain compared to reading +bytewise. And there is no harm ... + +Signed-off-by: Adrian Schmutzler +--- + target/linux/ar71xx/base-files/lib/ar71xx.sh | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/target/linux/ar71xx/base-files/lib/ar71xx.sh b/target/linux/ar71xx/base-files/lib/ar71xx.sh +index 2200069c64..2f3a72f732 100755 +--- a/target/linux/ar71xx/base-files/lib/ar71xx.sh ++++ b/target/linux/ar71xx/base-files/lib/ar71xx.sh +@@ -16,7 +16,7 @@ ar71xx_get_mtd_offset_size_format() { + dev=$(find_mtd_part $mtd) + [ -z "$dev" ] && return + +- dd if=$dev bs=1 skip=$offset count=$size 2>/dev/null | hexdump -v -e "1/1 \"$format\"" ++ dd if=$dev iflag=skip_bytes bs=$size skip=$offset count=1 2>/dev/null | hexdump -v -e "1/1 \"$format\"" + } + + ar71xx_get_mtd_part_magic() { +@@ -390,7 +390,7 @@ tplink_pharos_v2_get_model_string() { + part=$(find_mtd_part 'product-info') + [ -z "$part" ] && return 1 + +- dd if=$part bs=1 skip=4360 count=64 2>/dev/null | tr -d '\r\0' | head -n 1 ++ dd if=$part iflag=skip_bytes bs=64 skip=4360 count=1 2>/dev/null | tr -d '\r\0' | head -n 1 + } + + ar71xx_board_detect() { +-- +2.11.0 +