Commit Graph

4309 Commits

Author SHA1 Message Date
Simon Glass
7dc55435b2 patman: Pass the alias dict into gitutil.build_email_list()
Rather than accessing the settings module in this function, require the
alias dict to be passed in.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
e70fdbd7e4 patman: Untangle settings from gitutil
The gitutil module is supposed to be independent from patman but one
piece was missed in the series which separated them.

Move the settings setup out of gitutil

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
f9b84f9141 patman: Drop the old status code
Now that we have the new asyncio implementation, drop the unused
ThreadPoolExecutor code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
605717405f patman: Use a pager when outputting status
The pager makes it easier to review large number of comments. Add it
around the output.

Patman normally sends multiple concurrent requests to the patchwork
server, as this is faster. Provide an option to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
52aef33f95 patman: Provide an option to run in single-threaded mode
Patman normally sends multiple concurrent requests to the patchwork
server, as this is faster. Provide an option to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
45f4f62191 patman: Switch over to asyncio
Now that all the pieces are in place, switch over from using an executor
to using asyncio.

While we are here, import defaultdict to avoid needing to specify its
module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
39f258494f patman: Use defaultdict directly
Import defaultdict to avoid needing to specify its module in multiple
places. Fix an extra blank line while here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
0fb0b46200 patman: Add more information to Patch
The cover letter has some information on each patch, so allow this to be
stored in the Patch object.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
7052173fae patman: Show patches in yellow
When comments are shown below patches it can be hard to see the patch
subject. Use yellow instead of blue, since it stands out better.

Pass the colour object into show_responses() rather than creating a new
one, since that is tidier.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
289f4242c5 patman: Simplify test_find_new_responses()
This test uses the find_new_responses() function which combines
accessing patchwork and processing the results.

Since the test is aimed at testing processing, adjust it to call
process_reviews() instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
3fd99e2177 patman: Adjust how the fake request() function is provided
Instead of passing the URL and function to each call, put the fake
into the Patchwork object instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
3c5f27ce12 patman: Split out the processing of find_new_responses()
This function does patchwork calls and then processes the results. Move
the processing out so we can (later) test it separately.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
d63b075062 patman: Split up check_and_show_status()
This function has three phases:

- collecting things from patchwork
- doing some processing
- showing the results to the user / creating a branch

Refactor into two functions so we can eventually have the patchwork part
fully separated out.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
e2a991398d patman: Rename check_patchwork_status()
This function actually shows the status and does some other things.

Rename it to better reflect its purpose.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
d65490650f patman: Add reading of series and patch status
Expand the patchwork module so that it can match the current
requirements of the 'patman status' command, i.e. reading the state of a
series and the patches associated with it.

Since the format of each patchwork response is a little tricky to
understand, add examples in comments at the top of each function.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
24776ab276 patman: Move Patch and Review to patchwork module
These relate to information obtained from the patchwork server, so move
their definition into the new patchwork module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
aef44950db patman: Create a module for handling patchwork
At present the patchwork implementation is very simple, just consisting
of a function which calls the REST API.

We want to create a fake patchwork for use in tests. So start a new
module to encapsulate communication with the patchwork server.

Use asyncio since it is easier to handle lots of concurrent requests
from different parts of the code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
31d458b35d CI: Add a dependency file for patman
Now that patman has an unusual dependency, add a requirements.txt file
and use it in CI

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
770602195c patman: Move code for sending into its own module
The control module includes much of the implementation of patman's
'send' feature. As a first step to separating this out, move the sending
code into its own file.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
eca4653835 patman: Support extra test features
Provide support for the -X flag, which preserves the working directory
used by tests. Also support -N which shows captured output for tests.

Finally, allow selection of a particular test to run.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
6e64ae22c2 patman: Tidy up the start-up code
Much of this was written before the other Python tools and they have
evolved. Make a few updates:

- Rather than calling sys.exit(), return the exit code from the control
  module and use it in __main__
- Set up tout as it is used in some places
- We now have quite a few tests, so update the comment about that

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
f5ec4f05f9 patman: Rename test_basic() in test_checkpatch
This currently has the same name as a test in func_test.py so it isn't
possible to select one or the other with 'patman test test_basic': both
are executed.

Rename this one to avoid confusion.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
5053549532 patman: Correct a stray quote
A quote character was added a few years ago, but was not intended. Drop
it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
a61635d2af tools: Plumb in capture control
Add control of capturing output into u_boot_pylib and the tools which
use it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
9dee86cf53 u_boot_pylib: Allow disabling colour output
When running tests there is no situation in which we want ANSI output as
it makes it much harder to see what is going on in logs, tests, etc.

Provide a way to disable this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
55342144af u_boot_pylib: Avoid concurrent execution of only one test
There is no point in spinning up multiple processes if there is only one
test to execute. Add a check for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
1f18e8798f u_boot_pylib: Add a pager
It is useful to have a pager when outputting a lot of text. Add support
for this in the terminal library, making use of a context manager.

Also add a function to indicate whether the output device is a terminal
or not, while we are here, to avoid duplicating this code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
f74c6f9a81 u_boot_pylib: Improve ANSI-colour output with backgrounds
The current implementation does not handle background colours very well:

- It outputs an incorrect code in some cases, leading to wrong colours
- Some functions lack a control for the background

Tidy this up so that background colours can be used in more places.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
b697480f19 u_boot_pylib: Allow control of capturing
Tests often capture output so they can check it. This means that if the
test fails it is not easy to see what the output actually was.

Add a -N flag which writes out the output after it has been captured.
This is not a perfect solution but it is simple and seems to work well
in practice.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
84f3deaa99 binman: Correct a typo in the toolpath comment
Fix the 'ist' typo in the comment for TestFunctional

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
ae3695f691 patman: Move capture_sys_output() into terminal and rename
This function is sometimes useful outside tests. Also it can affect how
terminal output is done, e.g. whether ANSI characters should be emitted
or not.

Move it out of the test_util package and into terminal.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
6330f94a35 patman: Move all non-test logic into control module
It is easier for tests if the top-level control logic is all in one
module. Create a new do_patman() function to handle this. Move the
existing code into it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:41 +01:00
Simon Glass
92d5d1e1cd binman: x86: Write skip-at-start when end-at-4gb is used
The end-at-4gb property implies a value for skip-at-start so add it into
the output FDT so that U-Boot can read it.

Now that skip-at-start is implemented, we can drop the workarounds used
in the x86 code to obtain the correct image-pos value.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 10:07:39 +01:00
Simon Glass
c3d3534806 buildman: Add a way to build a particular target
At present buildman only supports building the default target. Generally
this is what is wanted, but in some cases boards erroneously have a
different target for product extra files.

Add a --target option to help. Also add a comment indicating which
letters are free for new options.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27 09:57:06 +01:00
Heinrich Schuchardt
285d265f7d Dockerfile: use lz4 instead of lz4-tools
Since Ubuntu Jammy lz4-tools is only a virtual package which pulls in
lz4 as dependency.

Update documentation too.

Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
2025-05-09 15:08:09 -06:00
Alice Guo
5f28a6599f tools: imx8image: add i.MX95 support
i.MX95 uses binman to invoke mkimage to create image container. 2 image
containers are needed currently. The first one is composed of
ahab-container.img, LPDDR firmware images, OEI images, System Manager
image and u-boot-spl.bin. The second one is consisted of ARM Trusted
firmware and u-boot.bin.

Because DDR OEI image and LPDDR firmware images have to be packaged
together and named as m33-oei-ddrfw.bin by binman, so imx9_image.sh does
not check if m33-oei-ddrfw.bin exists.

When using "make imx95_19x19_evk_defconfig; make", imx9_image.sh will
delete the line for u-boot.bin in container.cfg. In fact, binman is
always called after the u-boot.bin is built, so imx9_image.sh does not
check if u-boot.bin exists.

Signed-off-by: Alice Guo <alice.guo@nxp.com>
2025-05-03 16:55:32 -03:00
Alice Guo
0318c26c2f binman: add a new entry type for packing DDR PHY firmware images
i.MX95 needs to combine DDR PHY firmware images and their byte counts
together, so add a new entry type nxp-header-ddrfw for this requirement.

Signed-off-by: Alice Guo <alice.guo@nxp.com>
2025-05-03 16:55:32 -03:00
Tom Rini
d75998b476 Docker, CI: Add vexpress_fvp / vexpress_fvp_bloblist support
This adds the vexpress_fvp and vexpress_fvp_bloblist platforms to the
list of platforms we test via emulator in CI. In order to do this we
need to first have our container runtime have TF-A builds for the
vexpress_fvp platform, both with and without transfer list support as
well as installing "telnet" so that we can access console. In the CI
files we check for the existence of /opt/tf-a/${TEST_PY_BD} and if
found, copy bl1.bin and fip.bin to /tmp and set the variables so that we
can later run FVP to run.

Note that we currently disable the hostfs (semihosting) tests as they
trigger a bug in FVP. This has been reported upstream, and can be
enabled when fixed.

Reviewed-by: Harrison Mutai <harrison.mutai@arm.com>
Acked-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
2025-04-29 11:40:40 -06:00
Carlos López
2c529839a9 mkimage: fix option parsing segfault
getopt_long() expects a NULL-terminated list of structures. The current
list in mkimage does not have a zero-filled structure at the end, which
can cause getopt_long() to walk past the end of the array when passing
an unknown option, causing a segmentation fault.

As a reproducer, the following command causes a segmentation fault
(tested in Debian 12):

    mkimage --foobar

Signed-off-by: Carlos López <carlos.lopezr4096@gmail.com>
2025-04-28 13:25:59 -06:00
Tom Rini
efd00b0345 python: Use and refer to the venv module rather than virtualenv
Using some form of sandbox with Python modules is a long standing best
practice with the language. There are a number of ways to have a Python
sandbox be created. At this point in time, it seems the Python community
is moving towards using the "venv" module provided with Python rather
than a separate tool. To match that we make the following changes:

- Refer to a "Python sandbox" rather than virtualenv in comments, etc.
- Install the python3-venv module in our container and not virtualenv.
- In our CI files, invoke "python -m venv" rather than "virtualenv".
- In documentation, tell users to install python3-venv and not
  virtualenv.

Signed-off-by: Tom Rini <trini@konsulko.com>
2025-04-24 15:37:27 -06:00
Xuhui Lin
ec8c6fb0a6 rockchip: mkimage: Add rk3576 support
Add support for rk3576 package header in mkimage tool.

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Kever Yang <kever.yang@rock-chips.com>
2025-04-23 22:12:05 +08:00
Yifeng Zhao
68424ac83a rockchip: mkimage: Add support for RK3528
Add support for generating Rockchip Boot Image for RK3528.

Similar to RK3568, the RK3528 has 64 KiB SRAM and 4 KiB of it is
reserved for BootROM.

Signed-off-by: Yifeng Zhao <yifeng.zhao@rock-chips.com>
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Kever Yang <kever.yang@rock-chips.com>
2025-04-23 22:12:04 +08:00
Tom Rini
05a2b69841 buildman: Update to grabbing gcc-14.2.0 toolchains by default
With the switch to using GCC 14.2.0 in commit 001bac5f16 ("Dockerfile:
Update to gcc-14.2.0 and clang-18") in CI, we should make buildman match
this.

Signed-off-by: Tom Rini <trini@konsulko.com>
2025-04-15 11:55:17 -06:00
Tom Rini
a40fc5afae Merge patch series "binman: Check code-coverage requirements"
Simon Glass <sjg@chromium.org> says:

This series adds a cover-coverage check to CI for Binman. The iMX8 tests
are still not completed, so a work-around is included for those.

A few fixes are included for some other problems.

Link: https://lore.kernel.org/r/20250410124333.843527-1-sjg@chromium.org
2025-04-11 16:47:29 -06:00
Simon Glass
2911f2c1ee binman: Work around missing test coverage
The iMX8 entry-types don't have proper test coverage. Add a work-around
to skip this for now.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-11 14:29:52 -06:00
Jiaxun Yang
06a0d9eee8 binman: Workaround lz4 cli padding in test cases
Newer lz4 util is not happy with any padding at end of file,
it would abort with error message like:

Stream followed by undecodable data at position 43.

Workaround by skipping testCompUtilPadding test case and manually
strip padding in testCompressSectionSize test case.

Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Mattijs Korpershoek <mkorpershoek@kernel.org>
2025-04-11 14:29:52 -06:00
Simon Glass
702b7a3e2e binman: Drop algo check in CheckSetHashValue()
The CheckAddHashValue() function is always called before this one, so
the algorithm check is never used. Replace it with an assert to avoid a
coverage error.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-11 14:29:52 -06:00
Simon Glass
d664c29ec3 binman: fit: Drop unused code
The key-name-hint case is not tested so is presumably not used. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-11 14:29:52 -06:00
Simon Glass
0148be7cd4 binman: Drop GetRootSkipAtStart()
This method is not called anymore, so drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-11 14:29:52 -06:00
Simon Glass
a876295e1b binman: Exclude dist-packages and site-packages
Newer versions of the python3-coverage tool require a directory
separator before and after the directory name. Add this so that system
package are not included in the coverage report.

Signed-off-by: Simon Glass <sjg@chromium.org>
2025-04-11 14:29:52 -06:00