[feat](benchmark): replace load_param with load_param_mem #6286

0130w · 2025-08-26T19:25:39Z

Previous implementation required benchncnn to be run from the directory containing the model files. This often caused failures and inconvenience when executed from an arbitrary working directory.

This change embeds the model files directly into the executable by compiling them as C arrays. Consequently, benchncnn now becomes fully self-contained and can be run from any location without dependency on external model files.

github-actions · 2025-08-26T19:46:07Z

The binary size change of libncnn.so (bytes)

architecture	base size	pr size	difference
x86_64	15328640	15212648	-115992 😘
armhf	6229904	6206656	-23248 😘
aarch64	9527568	9524560	-3008 😘

tencent-adm · 2025-08-31T18:56:29Z

All committers have signed the CLA.

0130w · 2025-08-31T19:07:54Z

run

cd build
cmake -DCMAKE_BUILD_TYPE=Debug -DNCNN_VULKAN=ON -DNCNN_BUILD_EXAMPLES=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ..
make -j$(nproc)
cd benchmark
./benchncnn 4 8 0 -1 1

result before

./benchncnn 4 8 0 -1 1
loop_count = 4
num_threads = 8
powersave = 0
gpu_device = -1
cooling_down = 1
fopen squeezenet.param failed
network graph not ready
          squeezenet  min =    0.00  max =    0.00  avg =    0.00
fopen squeezenet_int8.param failed
...

after

loop_count = 4
num_threads = 8
powersave = 0
gpu_device = -1
cooling_down = 1
          squeezenet  min =   12.68  max =   13.12  avg =   12.94
     squeezenet_int8  min =    8.39  max =    9.11  avg =    8.88
           mobilenet  min =   16.69  max =   17.01  avg =   16.83
      mobilenet_int8  min =   15.41  max =   16.59  avg =   15.86
        mobilenet_v2  min =   16.33  max =   17.03  avg =   16.67
        mobilenet_v3  min =   13.08  max =   15.00  avg =   14.34
          shufflenet  min =   10.04  max =   12.69  avg =   10.88
       shufflenet_v2  min =   10.14  max =   13.40  avg =   11.20
             mnasnet  min =   12.67  max =   16.30  avg =   14.48
     proxylessnasnet  min =   14.56  max =   16.55  avg =   15.68
     efficientnet_b0  min =   23.54  max =   25.99  avg =   24.56
   efficientnetv2_b0  min =   25.69  max =   28.93  avg =   27.90
...

codecov-commenter · 2025-09-01T00:58:25Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.91%. Comparing base (5154f22) to head (41f463d).
⚠️ Report is 22 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6286      +/-   ##
==========================================
+ Coverage   95.89%   95.91%   +0.02%     
==========================================
  Files         841      844       +3     
  Lines      266337   267021     +684     
==========================================
+ Hits       255409   256122     +713     
+ Misses      10928    10899      -29

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull Request Overview

This PR replaces file-based model loading with memory-based loading in the benchncnn tool by embedding model parameter files directly into the executable as C arrays. This eliminates the dependency on external .param files and allows the benchmark to run from any directory.

Introduces a CMake macro to convert .param files into C header files containing hex data arrays
Updates benchncnn.cpp to use load_param_mem() instead of load_param() with embedded data
Normalizes model names by replacing hyphens with underscores to ensure valid C identifiers

Reviewed Changes

Copilot reviewed 6 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
cmake/ncnn_generate_model_data_header.cmake	Adds CMake macro to convert .param files to C arrays with validation
benchmark/model_param_spv_data.h.in	Template for including generated model parameter headers
benchmark/model_param_registry.h.in	Template for model parameter registry data
benchmark/benchncnn.cpp	Updates to use embedded model data with registry lookup function
benchmark/README.md	Updates documentation to reflect new param usage without .param suffix
benchmark/CMakeLists.txt	Integrates model data generation into build process

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

benchmark/benchncnn.cpp

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-01T02:24:39Z

benchmark/benchncnn.cpp

 #endif

-    if (fixed_path)
+    const char* model_param = NULL;


The variable model_param is declared but never used. This should be removed to clean up the code.

Suggested change

const char* model_param = NULL;

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-01T07:49:24Z

benchmark/benchncnn.cpp

-#else
-#define MODEL_DIR ""
-#endif
+    const char* model_param = NULL;


This variable model_param is declared but never used. It can be safely removed as it appears to be leftover from the refactoring.

Suggested change

const char* model_param = NULL;

Copilot · 2025-12-01T07:49:24Z

cmake/ncnn_generate_model_data_header.cmake

+    string(REGEX REPLACE "\n +" "\n" param_file_data ${param_file_data})
+
+    # remove empty line
+    string(REGEX REPLACE "\n\n" "\n" param_file_data ${param_file_data})


This regex replacement only removes one occurrence of double newlines at a time. If there are multiple consecutive empty lines (e.g., \n\n\n), only one will be removed, leaving \n\n. Consider using a loop or a regex pattern like "\n\n+" to remove all consecutive empty lines in a single pass.

Suggested change

string(REGEX REPLACE "\n\n" "\n" param_file_data ${param_file_data})

string(REGEX REPLACE "\n\n+" "\n" param_file_data ${param_file_data})

Copilot · 2025-12-01T07:49:24Z

cmake/ncnn_generate_model_data_header.cmake

+    # Use a macro to convert the contents of a single parameter file into memory content
+    file(READ ${PARAM_FILE} param_file_data)
+
+    # remove whitespace


The comment "remove whitespace" is misleading. The regex "\n +" only removes spaces that follow newlines (i.e., leading whitespace on lines), not all whitespace. Consider updating the comment to "remove leading whitespace from lines" or "strip indentation" for clarity.

Suggested change

# remove whitespace

# remove leading whitespace from lines

Copilot · 2025-12-01T07:49:24Z

cmake/ncnn_generate_model_data_header.cmake

+    # remove whitespace
+    string(REGEX REPLACE "\n +" "\n" param_file_data ${param_file_data})
+
+    # remove empty line


The comment "remove empty line" should be pluralized to "remove empty lines" for grammatical correctness, as the operation attempts to remove multiple occurrences of empty lines (though the current implementation only removes one pair at a time - see related bug comment on line 10).

Suggested change

# remove empty line

# remove empty lines

0130w added 3 commits August 26, 2025 19:12

[feat](benchmark): replace load_param with load_param_mem

a555f51

[docs](benchmark): update benchmark docs

f4999de

Merge branch 'master' into dev

1643c32

github-actions bot added the cmake label Aug 26, 2025

fix

41745c8

nihui requested a review from Copilot September 1, 2025 02:12

Copilot AI reviewed Sep 1, 2025

View reviewed changes

benchmark/benchncnn.cpp Outdated Show resolved Hide resolved

benchmark/benchncnn.cpp Outdated Show resolved Hide resolved

0130w added 3 commits November 30, 2025 21:52

Merge branch 'master' into dev

aae6ad8

refactor(benchmark): remove model registry and optimize param loading

74ae9d1

fix(benchmark): correct README param description

086e73c

nihui requested a review from Copilot December 1, 2025 02:23

Copilot started reviewing on behalf of nihui December 1, 2025 02:23 View session

Copilot finished reviewing on behalf of nihui December 1, 2025 02:24

Copilot AI reviewed Dec 1, 2025

View reviewed changes

refactor(benchmark): rename header file

41f463d

nihui requested a review from Copilot December 1, 2025 07:46

Copilot started reviewing on behalf of nihui December 1, 2025 07:47 View session

Copilot finished reviewing on behalf of nihui December 1, 2025 07:48

Copilot AI reviewed Dec 1, 2025

View reviewed changes

	string(REGEX REPLACE "\n\n" "\n" param_file_data ${param_file_data})
	string(REGEX REPLACE "\n\n+" "\n" param_file_data ${param_file_data})

[feat](benchmark): replace load_param with load_param_mem #6286

Are you sure you want to change the base?

[feat](benchmark): replace load_param with load_param_mem #6286

Conversation

0130w commented Aug 26, 2025

Uh oh!

github-actions bot commented Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tencent-adm commented Aug 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

0130w commented Aug 31, 2025

Uh oh!

codecov-commenter commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Aug 26, 2025 •

edited

Loading

tencent-adm commented Aug 31, 2025 •

edited

Loading

codecov-commenter commented Sep 1, 2025 •

edited

Loading