36,493 questions
3
votes
2
answers
192
views
Force the C compiler to read the object's memory
I duplicated the topic because (possibly) I made a mistake with the section of the site and initially posted in the wrong place: Ways to reliably read an object in C
I have a regular byte buffer ...
- 509
2
votes
1
answer
80
views
Freeing memory to OS after heavy function without forcing GC
I have a Python function that consumes a large amount of memory. When the function finishes, I want to release the memory and return the freed pages to the operating system by calling malloc_trim(0).
...
- 21
1
vote
1
answer
86
views
Cause of AlignmentMismatch error while using bytemuck?
I'm facing an AlignmentMismatch error while trying to cast a Vec<u8> to Vec<T>, where T is a struct that uses repr(C) and Pod + Zeroable.
I've also tried the slice road &[u8] -> &...
- 984
0
votes
0
answers
46
views
Taking advantage of memory contiguousness in HLSL
This is a bit of a slog so bare with me.
I'm currently writing a 3D S(moothed) P(article) H(ydrodynamics) simulation in Unity with a parallel HLSL backend. It's a Lagrangian method of fluid simulation,...
- 13
-3
votes
0
answers
41
views
How can I understand what drains memory in my Java app? [duplicate]
I suspect that my Java app running in tomcat is draining some memory, It is a Spring application with most stateless services and then I think the memory consumption should be low especially while no ...
- 9,609
0
votes
1
answer
63
views
Unable to find required GLIBC private symbols
I am trying to analyze a memory dump obtained with dotnet-dump from a Ubuntu container in WinDbg to find an unmanaged memory leak.
I have been able to load the dump, but whenever I try to run !heap -s,...
- 9,635
1
vote
0
answers
89
views
Why do I observe so high memory usage for copying a Parquet file with streaming using Polars with Rust?
Goal
I want to write a function in Rust using the Polars crate that does the following:
Copy a Parquet file from one location to another
Handle files larger than RAM
Not load the entire file into ...
- 457
Best practices
2
votes
6
replies
133
views
Memory accounting and quotas for subtasks in golang programs
If I have a server written in golang that processes requests containing user queries, is there any viable method to track and limit the memory used by any given user request?
The server process does ...
- 329k
2
votes
1
answer
118
views
Does `jax` compilation save runtime memory by recognizing array elements that are duplicated by indexing
Consider the example code:
from functools import partial
from jax import jit
import jax.numpy as jnp
@partial(jit, static_argnums=(0,))
def my_function(n):
idx = jnp.tile(jnp.arange(n, dtype=int)...
- 539
0
votes
0
answers
44
views
Confused by Valgrind Massif output - appears memory allocated under "free"
I'm attempting to use Massif to profile heap usage of an application. I'm running with the --pages-as-heap=true option and am confused by some output I'm seeing. Specifically I'm seeing allocation by ...
- 549
3
votes
3
answers
331
views
Trying to create Buffer overflow for an example for school in C++
I'm trying to do the very basics of a buffer overflow, nothing illegal at all. My understanding of the code below is that the overflowing buffer doesnt hold enough space for all the data being written,...
- 41
1
vote
1
answer
79
views
Handling cleanup for tasks which might be OOMKilled
I have some Python code running in k8s which in some cases is being OOMKilled and not leaving me time to cleanup which is causing bad behavior.
I've tried multiple approaches but nothing seems quite ...
- 175
3
votes
2
answers
157
views
.NET 8 console application slower after publish
I am confused why my application is somehow slower when I run it by executing the .exe file from the publish folder, compared to running it from Visual Studio debugger (F5).
I use .NET 8.0 and this is ...
- 586
0
votes
0
answers
39
views
java spring application - unhealthy heap behaviour
So i have a very relatively simple java application spring 2.3.7 on java 8 that microservice simply calls DB to save update or retrive base64 image string thats all.
So i have deployed that in my ...
0
votes
0
answers
43
views
Is there a performance difference between writing bytes to RAM instead of longs?
My single threaded program allocates and initializes a volatile block of memory on an unspecified hardware architecture.
It then writes into this block in a loop using a stride equal to the cache line ...
- 33k
0
votes
0
answers
47
views
H2O-3 Calling a lot of GC calls in java when trying to load CSV file with 16M rows
I am using h2o-3 java repo to load this frame but have been running into memory issues with constant GC pressure.
The actual frame size is 3.31 GB as per h2o logs, but the peak JVM usage comes to be ...
0
votes
1
answer
100
views
Duckdb Wasm limitation
I don't know how to check or increase the memory limitation of duckdb wasm.
I'm using chrome and I import some parquet into the browser, one of them has 234Mb of data
I did my research and the limit ...
- 1
-2
votes
1
answer
130
views
Is this a valid JavaScript approach for sorting large arrays of unique integers efficiently? [closed]
I’m experimenting with sorting large arrays of unique integers in JavaScript and came up with a simple approach. I’m curious if it’s a recognized pattern or if there are potential pitfalls I might be ...
- 1
0
votes
1
answer
61
views
Python moviepy V2 memory management
I am trying to combine a background video with captions generated word by word from a tts file. I am using nvenc for ffmpeg but my gpu's cuda version is too old to use it for whisper. The script works ...
- 23
1
vote
1
answer
133
views
Is address sanitizer supposed to catch this error?
I tried it with -fsanitize=address and it didn't show any error. Is this a problem with my address sanitizer or system, or is this normal? Also, how to catch these situations?
#include <iostream>...
- 16.3k
0
votes
1
answer
214
views
Nuxt 3 to 4 migration : will it fix memory issues?
I have a Nuxt 3 production app hosted on Scalingo (it's like Heroku but is a french company)
I have so many container crashes because of memory issues when a lot of people are browsing the website.
...
- 804
0
votes
1
answer
102
views
Why can't successfully kmalloc pages in this case?
See the below logs, A critical kernel driver cannot apply for a large block of continuous memory and then panic:
[29001.362425] kworker/u12:4: page allocation failure: order:4, mode:0x40dc0(GFP_KERNEL|...
1
vote
1
answer
106
views
Aws::ShutdownAPI crashes when called by shared_ptr
I am designing a library where I need AWS SDK for S3 and Kinesis as wrapped sub-components. In an executable, Aws::InitAPI() and Aws::ShutdownAPI() can be called once, so I am trying to design a ...
- 319
0
votes
0
answers
140
views
Does accessing the contents of the string after calling reserve causes UB? [duplicate]
From another thread I found that
indeed allocates enough storage to hold at least n elements, but it doesn't actually fill the container with any elements
If elements are already allocated why ...
- 27
3
votes
0
answers
121
views
What is the best/correct way to handle shared memory objects in a heterogeneous multi-CPU embedded microprocessor context [closed]
I'm working on several projects that are targeting bare-metal multi-core microprocessors, with hardware provisions for shared memory.
In other words, the device has several separate CPU cores (of ...
- 5,943
0
votes
0
answers
94
views
move_pages() only is effective temporarily in C++
I'm trying to write NUMA-aware codes, and i encountered some unexpected behavior. I decide to use move_pages() to test if i create the correct NUMA placement but the result is very confusing.
for (...
- 11
1
vote
1
answer
113
views
Which resources of a modern x86 CPU core are occupied by memory transactions in flight?
I want to clarify how modern x86 architectures handle the latency of memory transactions that go all the way to DRAM. Specifically, which resources (which queues) get occupied waiting for the memory ...
- 4,808
0
votes
0
answers
23
views
Does alloc_calls of slab_info will decrease if the memory is freed?
I found the count of an API in alloc_calls is not increased over time and decreased if I free allocated memory. Is this true?
So if I unload the driver, the API should disappear in the alloc_calls ...
- 1
1
vote
1
answer
124
views
Reading another process' memory in C not returning results after multiple queries
I'm trying to read a variable from another process' memory in C and upon first query, for which I'm using VirtualQueryEx function, some potentially matching values are returned. However, when I change ...
- 63
2
votes
2
answers
172
views
How CPU read memory in alignment?
I have read a post about Why we need memory alignment.
I mostly focus the effects on the performance when CPU read memory.
The CPU always reads at its word size (4 bytes on a 32-bit processor), so ...
- 33
5
votes
1
answer
168
views
Zero-initialized array keeps having non-zero values [closed]
I am encountering something extremely strange on my system right now.
#include <cstdint>
#include <iostream>
constexpr size_t N = 1l<<31;
int main() {
auto buf = new int64_t[N]{...
- 2,123
0
votes
1
answer
105
views
Grails app + Tomcat + Spring Boot - Memory usage is increasing while the application is idle
I started to test Docker to run my application and I had noticed my memory consume when using "docker stats". I see the memory used from the container increasing like, every f5 on my page or ...
- 9
0
votes
0
answers
129
views
Python logging MemoryHandler leaks memory?
In a main.py file I have following code:
import time
from v1 import log, ServiceLogger
from logging.handlers import MemoryHandler
class Service:
h = MemoryHandler(1000)
id = None
log = ...
- 11
-2
votes
2
answers
136
views
Facing issues with Structs and malloc in an assignment question [closed]
This was a question in a C Programming assignment given to me as a part of last week's assessment. Requesting everyone to kindly explain what needs to be done and how it needs to be done. Kindly note ...
0
votes
0
answers
64
views
Can you give me advice on memory issue from my flutter project?
The app I developed shows the highest memory usage on my device—even more than YouTube, Instagram, or other SNS apps—despite not handling videos or large image data.
I’ve searched extensively for ...
- 1,314
2
votes
1
answer
129
views
Weird problems with Zig Array List(s)
In the below code, I have the function "get_specific_files". The objective of this function is to check if certain filenames contain some word, and then return all the files that contain ...
- 53
0
votes
2
answers
252
views
Combining two huge text files and removing duplicates
The Specific Problem
I am starting with two gigantic input text files containing lots of lines with a single string value on each line. I'd like to write a Java program to combine the contents of ...
- 1,053
0
votes
0
answers
48
views
What can corrupt memory in Julia -- "probably modified after bring freed?"
I am very new to Julia. I am working on a project written in Julia and I am using the following script:
import Pkg
Pkg.activate(@__DIR__)
project_root = normpath(joinpath(@__DIR__, "..", &...
0
votes
0
answers
97
views
Flutter: Memory Usage Issue
I’m building a Flutter app, and some users are receiving the following device notification:
“This app uses a lot of memory, so your phone is slow. Switch this feature to ultra power saving so your ...
- 1,314
1
vote
0
answers
58
views
Is it possible to reduce the memory requirements for (I)FFT by chunking?
I'm trying to run an IFFT on a very long signal (10^9 data points) and I'm running into RAM constraints. I'd like to be able to chop up the signal and run it in batches (as I am not time constrained ...
- 454
0
votes
0
answers
80
views
hight memory usage problem in MVC web app
in mvc web app, first chrome memory snapshot is ~ 51mb
after i call ajax to re render partialView,
second snapshot take double or more! ~ 100bm.
and in every call partialview memory usage get larger ...
- 131
1
vote
1
answer
200
views
Why does polars kept killing the python kernel when joining two lazy frames and collecting them?
I have one dataframe: bos_df_3 that has about a 30k+ rows and another, taxon_ranked_only, with 6 million when I tried to join them using:
matching_df = (
pl.LazyFrame(bos_df_3)
.join(
other=...
- 436
1
vote
0
answers
59
views
Error with memory while running a small size file
When running the following code:
events <- read.csv("C:\\Users\\jsh274\\OneDrive - University of Canterbury\\6. Data\\3. Scvenja_data\\3. Jeniya\\1. Analysis\\Output\\3.1 ...
- 31
2
votes
1
answer
285
views
Laravel Filament Admin Panel: "Allowed memory size exhausted" error
I'm working on a Laravel project using Filament for the admin dashboard. Initially, everything worked fine with a few resources. But after adding a fourth resource, the panel started throwing memory ...
- 23
1
vote
2
answers
102
views
Rust borrow checker: "cannot borrow `...` as immutable because it is also borrowed as mutable [E0502] immutable borrow occurs here"
I wanted to implement the Merge Sort algorithm, but I am having trouble with the borrow checker rules.
The compiler gives me the hint:
cannot borrow *lst as immutable because it is also borrowed as ...
- 105
2
votes
2
answers
195
views
Is it safe to .pop_back() from an std::vector in order to avoid pointers/memory shifting?
I have this class:
class Socket
{
[...]
pollfd* fdPtr;
};
And I have a Manager class that creates the actual pollfd objects and adds them to a std::vector named pollFdsVec, to then poll on ...
- 45
2
votes
1
answer
150
views
Linux: Low-level management of virtual memory - collaboration between processes
If I were to mmap() a very large file (for read-only access) pages would be read-in, by the kernel, after a page fault indicates that the page is not already cached in RAM. Pages will remain cached ...
- 2,056
0
votes
0
answers
208
views
Jax unable to allocate memory despite memory being free?
I have been having issues allocating memory on the GPU with jax. I am running on a cluster that gives me access to a RTX6000 (24GiB vram) which jax is attempting to allocate to.
Output of jax....
- 305
3
votes
0
answers
140
views
Memory usage accumulates in simple loop until PC crashes
I have recently experienced major crashes on my computer after executing some Python code of mine. My PC would freeze and sometimes show a bluescreen after a few rounds of a seemingly unproblematic ...
- 139
0
votes
3
answers
235
views
Array of pointers to char looks weird in memory [closed]
I have an array of char pointers (C-style string). As far as I know C-style string containing N characters occupy N + 1 bytes in memory since it also has a null character '\0' at the end terminating ...
- 173