Hi,

This is the third version. In this version, Some codes are fixed
and more description and note are added. I would like to thank David Sterba
for his review.

The Last patch[5/5] of the patch set is for making x86 and arm default to
LZ4-compressed for testing the LZ4 code in the linux-next.
It was requested by Andrew Morton in the patch set v2.

Currently, A preliminary version of LZ4 de/compression tool is supported.
However, It is expected that we will have a tool with more features
once its format is finished.

LZ4 compression tool is available at
http://code.google.com/p/lz4/source/checkout.

Thanks,
Kyungsik


Change log: v2
- Clean up code
- Enable unaligned access for ARM v6 and above with
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
- Add lz4_decompress() for faster decompression with
uncompressed output size
- Use lz4_decompress() for LZ4-compressed kernel during
boot-process
- Apply -Os to decompress.o to improve decompress
performance during boot-up process
Change log: v3
- Prevent double evaluation by using an inline function
- Add LZ4 description and note for uncompressed chunk size issue
- Fix indentation error


Kyungsik Lee (5):
decompressor: Add LZ4 decompressor module
lib: Add support for LZ4-compressed kernel
arm: Add support for LZ4-compressed kernel
x86: Add support for LZ4-compressed kernel
Kconfig: Make x86 and arm kernels default to the LZ4-compressed

arch/arm/Kconfig | 1 +
arch/arm/boot/compressed/.gitignore | 1 +
arch/arm/boot/compressed/Makefile | 6 +-
arch/arm/boot/compressed/decompress.c | 4 +
arch/arm/boot/compressed/piggy.lz4.S | 6 +
arch/x86/Kconfig | 1 +
arch/x86/boot/compressed/Makefile | 5 +-
arch/x86/boot/compressed/misc.c | 4 +
include/linux/decompress/unlz4.h | 10 ++
include/linux/lz4.h | 51 ++++++
init/Kconfig | 19 +-
lib/Kconfig | 7 +
lib/Makefile | 2 +
lib/decompress.c | 5 +
lib/decompress_unlz4.c | 187 +++++++++++++++++++
lib/lz4/Makefile | 1 +
lib/lz4/lz4_decompress.c | 326 ++++++++++++++++++++++++++++++++++
lib/lz4/lz4defs.h | 94 ++++++++++
scripts/Makefile.lib | 5 +
usr/Kconfig | 9 +
20 files changed, 740 insertions(+), 4 deletions(-)
create mode 100644 arch/arm/boot/compressed/piggy.lz4.S
create mode 100644 include/linux/decompress/unlz4.h
create mode 100644 include/linux/lz4.h
create mode 100644 lib/decompress_unlz4.c
create mode 100644 lib/lz4/Makefile
create mode 100644 lib/lz4/lz4_decompress.c
create mode 100644 lib/lz4/lz4defs.h

--
1.8.1.1

On Tue, 5 Mar 2013 20:47:31 +0900 Kyungsik Lee <kyungsik.lee@...> wrote:

This is the third version. In this version, Some codes are fixed
and more description and note are added. I would like to thank David Sterba
for his review.

The Last patch[5/5] of the patch set is for making x86 and arm default to
LZ4-compressed for testing the LZ4 code in the linux-next.
It was requested by Andrew Morton in the patch set v2.

Currently, A preliminary version of LZ4 de/compression tool is supported.
However, It is expected that we will have a tool with more features
once its format is finished.

What happened to the changelog? The earlier version at least had some
rudimentary benchmarking results, but now we don't even have that.

Someone should prepare the information explaining why we added this to
Linux, and I'd prefer that person be you rather than me! Certainly it
should include performance measurements - both speed and space. Also
it should capture our thinking regarding all the other decompressors,
explaining why we view it as acceptable to add yet another one.

Please, put yourself in the position of someone reading these commits
in 2017 wondering "why did they merge this". We should tell them.

On Tue, Mar 05, 2013 at 03:06:16PM -0800, Andrew Morton wrote:

On Tue, 5 Mar 2013 20:47:31 +0900 Kyungsik Lee <kyungsik.lee@...> wrote:

This is the third version. In this version, Some codes are fixed
and more description and note are added. I would like to thank David Sterba
for his review.

The Last patch[5/5] of the patch set is for making x86 and arm default to
LZ4-compressed for testing the LZ4 code in the linux-next.
It was requested by Andrew Morton in the patch set v2.

Currently, A preliminary version of LZ4 de/compression tool is supported.
However, It is expected that we will have a tool with more features
once its format is finished.

What happened to the changelog? The earlier version at least had some
rudimentary benchmarking results, but now we don't even have that.

Someone should prepare the information explaining why we added this to
Linux, and I'd prefer that person be you rather than me! Certainly it
should include performance measurements - both speed and space. Also
it should capture our thinking regarding all the other decompressors,
explaining why we view it as acceptable to add yet another one.

Sorry, It will have all the information you mentioned. I can reply to
this thread if you want.

Or I might need another patch(v4). I have one thing to be confirmed
regarding LZ4 format. One of the patch set needs to be fixed
if there is a change. After that I can include the information
in the changelog.

Please, put yourself in the position of someone reading these commits
in 2017 wondering "why did they merge this". We should tell them.

Thanks,
Kyungsik

Hello,

On Tue, Mar 05, 2013 at 03:06:16PM -0800, Andrew Morton wrote:

On Tue, 5 Mar 2013 20:47:31 +0900 Kyungsik Lee <kyungsik.lee@...> wrote:

This is the third version. In this version, Some codes are fixed
and more description and note are added. I would like to thank David Sterba
for his review.

The Last patch[5/5] of the patch set is for making x86 and arm default to
LZ4-compressed for testing the LZ4 code in the linux-next.
It was requested by Andrew Morton in the patch set v2.

Currently, A preliminary version of LZ4 de/compression tool is supported.
However, It is expected that we will have a tool with more features
once its format is finished.

What happened to the changelog? The earlier version at least had some
rudimentary benchmarking results, but now we don't even have that.

Someone should prepare the information explaining why we added this to
Linux, and I'd prefer that person be you rather than me! Certainly it
should include performance measurements - both speed and space. Also
it should capture our thinking regarding all the other decompressors,
explaining why we view it as acceptable to add yet another one.

Please, put yourself in the position of someone reading these commits
in 2017 wondering "why did they merge this". We should tell them.

Sorry for the inconvenience regarding changelog. Another patch(v4)
is not required so this is the information you mentioned.
I'm not sure that I captured what we had discussed regarding all the other
decompressors properly.


Benchmark Results(PATCH v3)
Compiler: Linaro ARM gcc 4.6.2

1. ARMv7, 1.5GHz based board
Kernel: linux 3.4
Uncompressed Kernel Size: 14MB
Compressed Size Decompression Speed
LZO 6.7MB 20.1MB/s, 25.2MB/s(UA)
LZ4 7.3MB 29.1MB/s, 45.6MB/s(UA)

2. ARMv7, 1.7GHz based board
Kernel: linux 3.7
Uncompressed Kernel Size: 14MB
Compressed Size Decompression Speed
LZO 6.0MB 34.1MB/s, 52.2MB/s(UA)
LZ4 6.5MB 86.7MB/s
- UA: Unaligned memory Access support
- Latest patch set for LZO applied


This patch set is for adding support for LZ4-compressed Kernel.
LZ4 is a very fast lossless compression algorithm and it also features
an extremely fast decoder [1].

But we have five of decompressors already and one question which does arise,
however, is that of where do we stop adding new ones? This issue had been
discussed and came to the conclusion [2].
Russell King said that
we should have:
- one decompressor which is the fastest
- one decompressor for the highest compression ratio
- one popular decompressor (eg conventional gzip)
If we have a replacement one for one of these, then it should do exactly that:
replace it.

The benchmark shows that an 8% increase in image size vs a 66% increase
in decompression speed compared to LZO(which has been known as the fastest
decompressor in the Kernel). Therefore the "fast but may not be small"
compression title has clearly been taken by LZ4 [3].

[1] http://code.google.com/p/lz4/
[2] http://thread.gmane.org/gmane.linux.kbuild.devel/9157
[3] http://thread.gmane.org/gmane.linux.kbuild.devel/9347


Thanks,
Kyungsik