If this is happening to you right now, your first move is probably to go re-check the shared memory setup. Right name, right size, right permissions. That’s not where the bug is. It’s not that the reader wasn’t allowed to look at the memory. It’s that it looked too early.

(There’s a different bug that looks similar from far away: the reader runs before the writer has even created the segment, so shm_open() itself returns -1. That one’s boring, you fix it by checking your return values and refusing to continue on a bad fd. It’s not what’s happening here, and I’m not going to spend more time on it, because the interesting case, and the one that actually costs people hours, is the one where every single syscall succeeds and the crash happens anyway.)

What’s actually sitting in that memory

Here’s the setup. The writer creates the shared memory object and sizes it:

int fd = shm_open("/shared_memory", O_CREAT | O_RDWR, 0666);
ftruncate(fd, 4096);
void *addr = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);

ftruncate() does two things here: it sets the segment to 4096 bytes, and since this is a freshly created object, it zero-fills those bytes on the way there. That’s not a courtesy, it’s the kernel refusing to hand your process leftover contents of some physical page it doesn’t know the history of. Fine. But “zero-filled” is not “contains the message the writer is about to send.” It’s zeros. Placeholder bytes that happen to be sitting there because something had to be.

And that’s the good case. If the segment already existed from an earlier run of your program, and you didn’t clean it up, ftruncate() to the same size it already has does nothing at all. No zeroing, no reset. Whatever was in there from last time is still in there, and it is very much not guaranteed to look like zeros.

Say the writer and reader agree on this layout:

typedef struct payload {
    size_t length;
    char buffer[];
} payload;

The writer’s job is eventually to set length and copy a string into buffer. The reader’s job is to read length, then copy that many bytes back out:

size_t length = *(size_t *)addr;
char string[length];
memcpy(string, (char *)addr + sizeof(size_t), length);

Now picture the two processes starting at roughly the same time, which is the normal case, not some pathological edge case you have to work hard to trigger:

writer                              reader
------                              ------
shm_open(O_CREAT)
ftruncate()
mmap()
                                     shm_open()   <- segment exists, succeeds
                                     mmap()       <- mapping is valid, succeeds
                                     length = *(size_t *)addr   <- reads zeros or stale bytes
data->length = length
memcpy(data->buffer, ...)

Nothing in that reader column failed. There’s no line in there you could wrap in an if and catch. The reader just happened to read length a few instructions before the writer got around to setting it, and there was nothing stopping it from doing so.

So what does the reader actually get? If the segment was freshly zeroed, length comes out as 0. char string[length] is a zero-length VLA, memcpy() copies nothing, and you get an empty string. Weird output, but the process survives.

If the segment had stale bytes from a previous run instead, length can be anything a size_t holds. Say it comes out enormous. char string[length] isn’t a heap allocation that can fail gracefully and hand you NULL. It’s a stack pointer bump, done on the spot, at the point of declaration. If that number is big enough, you blow through the rest of the stack before you’ve written a single byte of the string, and the process dies right there, on a line that doesn’t even mention memcpy().

And if the garbage length is large but not stack-shattering large, memcpy() goes ahead and reads that many bytes starting at addr + sizeof(size_t). Run past the end of the 4096-byte mapping and you get a segfault from touching unmapped memory. Or you don’t run quite that far, and instead you silently read whatever else happens to be mapped nearby, and hand your caller a string that isn’t garbage-looking at all, it’s just wrong. That second version is worse, because nothing crashes and nothing looks broken.

The thing worth sitting with here is that shm_open() returned a valid fd and mmap() returned a valid pointer both times, in both the empty-string case and the stack-overflow case. The syscalls did their job. The bug is entirely downstream of them, in the assumption that a successful mapping means there’s something meaningful behind it.

sleep() is not a fix

Once you’ve figured out it’s a timing problem, the first instinct is almost always some version of “just wait a little before reading.” None of the usual ways of doing that actually close the race.

The most common one is a retry loop with sleep() in it: poll length, and if it still looks unset, sleep and check again. This appears to work, most of the time, because your writer probably finishes in a few milliseconds and your sleep is probably a full second, so in practice you never catch it in the act. That’s not the same thing as it being fixed. It’s a race where you’ve made the reader’s side slower, which changes the odds without touching the actual problem. Put the machine under load, or add anything at all to the writer’s startup, like reading a config file or allocating a buffer, and your sleep duration stops being generous enough. You’re back to the original bug, just less often, which honestly might be worse, because now it’s the kind of bug that only shows up in production.

A slightly smarter-sounding version checks whether the memory is still all zeros before reading it. This has the same timing problem as the sleep loop, plus a correctness problem of its own: zero is a perfectly legitimate value. If the writer’s real payload happens to produce a length of 0, or the first eight bytes of a valid message happen to be zero, your “is it ready yet” check can’t tell that apart from “not ready yet.” And as covered above, if the segment wasn’t freshly zeroed, “non-zero” doesn’t mean “written by this run” either. Leftover bytes from an old run can easily be non-zero, so this check can happily report “looks ready” on a segment the current writer hasn’t touched at all.

And then there’s just assuming the writer runs first because you start it first. There’s no guarantee anywhere that backs this up. Process creation order and scheduler decisions aren’t something POSIX promises you control over, and even setting that aside, “started first” isn’t “finished writing.” If the writer does anything at all before it gets to the actual write, that’s a window where a reader that technically started later can still get there first.

All three of these are trying to guess readiness from either the contents of the memory or the passage of time. What you actually want is for the writer to tell you, explicitly, “I’m done,” and for the reader to be physically unable to proceed until it hears that. That’s what a semaphore gives you, and it’s the only one of these that isn’t a guess.

A semaphore that only goes one way

If you’ve used semaphores before, it was probably as a mutex: a binary semaphore that starts at 1, where a thread calls sem_wait() to grab it, does some work, and calls sem_post() to hand it back. Both sides call both functions. It’s symmetric, and it’s protecting a critical section.

That’s not what we’re building here, and thinking of this as “a lock” will make the rest of it confusing. What we want is a readiness signal, and it differs in two specific ways:

• It starts at 0, not 1. Zero means “nothing to report yet.” A lock starts at 1 because the resource is free from the beginning; a readiness signal starts at 0 because the data isn’t ready from the beginning, and shouldn’t be treated as ready until someone says otherwise.
• The two sides don’t do the same thing. The writer only ever calls sem_post(). The reader only ever calls sem_wait(). Nobody waits and posts around the same operation the way a mutex would have both sides do. It only goes one direction: the writer speaks once, the reader listens once.

The mechanics: sem_wait() decrements the semaphore’s counter. If the counter’s already at 0, it doesn’t go negative, it just blocks, parking the calling process until something else bumps the count back up. This is a genuine suspension, handled by the kernel, not a spin loop and not a sleep() in disguise. sem_post() increments the counter, and if there’s a process sitting in sem_wait(), it wakes exactly one of them.

Put that back into our reader and writer. The semaphore starts at 0. The reader hits sem_wait() before it has touched a single byte of the mapped memory, sees a count of 0, and blocks right there. It cannot execute the next line. It’s not choosing to wait, it’s stuck. Meanwhile the writer sets length, copies the string into buffer, and only after both of those are done does it call sem_post(). That’s what bumps the count and wakes the reader up. The reader’s sem_wait() returns, and only now, after the writer is provably finished, does the reader go read length and buffer.

Go back to the diagram from earlier. There’s no version of it anymore where the reader’s read of length can land before the writer’s write of length, because the line that does the reading is now behind a wall the writer built, and the writer only takes that wall down once it’s actually done.

Here’s the fix, in full

Complete writer and reader, error-checked this time. The writer owns creating both the shared memory object and the semaphore; the reader just opens what’s already there.

writer.c

#include <fcntl.h>
#include <semaphore.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <unistd.h>

#define SHM_NAME "/shared_memory"
#define SEM_NAME "/semaphore"
#define SHM_SIZE 4096

typedef struct payload {
    size_t length;
    char buffer[];
} payload;

int main(void) {
    int fd = shm_open(SHM_NAME, O_CREAT | O_RDWR, 0666);
    if (fd == -1) {
        perror("shm_open");
        exit(EXIT_FAILURE);
    }

    if (ftruncate(fd, SHM_SIZE) == -1) {
        perror("ftruncate");
        exit(EXIT_FAILURE);
    }

    void *addr = mmap(NULL, SHM_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    if (addr == MAP_FAILED) {
        perror("mmap");
        exit(EXIT_FAILURE);
    }

    /* Initial value 0: nothing is ready until we say so. */
    sem_t *sem = sem_open(SEM_NAME, O_CREAT, 0666, 0);
    if (sem == SEM_FAILED) {
        perror("sem_open");
        exit(EXIT_FAILURE);
    }

    const char *message = "Hello world from Process A";
    size_t length = strlen(message) + 1;

    payload *data = (payload *)addr;
    data->length = length;
    memcpy(data->buffer, message, length);

    /* Only signal readiness after the payload is fully written. */
    if (sem_post(sem) == -1) {
        perror("sem_post");
        exit(EXIT_FAILURE);
    }

    printf("writer: wrote %zu bytes and signaled the reader\n", length);

    munmap(addr, SHM_SIZE);
    close(fd);
    sem_close(sem);
    return 0;
}

reader.c

#include <fcntl.h>
#include <semaphore.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <unistd.h>

#define SHM_NAME "/shared_memory"
#define SEM_NAME "/semaphore"
#define SHM_SIZE 4096

typedef struct payload {
    size_t length;
    char buffer[];
} payload;

int main(void) {
    int fd = shm_open(SHM_NAME, O_RDWR, 0666);
    if (fd == -1) {
        perror("shm_open");
        exit(EXIT_FAILURE);
    }

    void *addr = mmap(NULL, SHM_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    if (addr == MAP_FAILED) {
        perror("mmap");
        exit(EXIT_FAILURE);
    }

    sem_t *sem = sem_open(SEM_NAME, 0);
    if (sem == SEM_FAILED) {
        perror("sem_open");
        exit(EXIT_FAILURE);
    }

    printf("reader: waiting for data...\n");

    /* Blocks here until the writer calls sem_post(). No memory is
       touched before this line returns. */
    if (sem_wait(sem) == -1) {
        perror("sem_wait");
        exit(EXIT_FAILURE);
    }

    payload *data = (payload *)addr;
    size_t length = data->length;

    char *string = malloc(length);
    if (!string) {
        perror("malloc");
        exit(EXIT_FAILURE);
    }
    memcpy(string, data->buffer, length);

    printf("reader: got %zu bytes: %s\n", length, string);

    free(string);
    munmap(addr, SHM_SIZE);
    close(fd);
    sem_close(sem);
    return 0;
}

The writer passes O_CREAT to both shm_open() and sem_open(); the reader passes neither, so if it happens to run before the writer has created either one, it fails immediately with a plain ENOENT instead of quietly creating something itself. The shm_open()/mmap() checks are what catch that case specifically; sem_wait() is what catches the one this whole article is about, where the segment exists but isn’t populated yet. The two don’t overlap and neither one covers for the other.

One more thing about the reader: it copies the payload into a heap buffer sized with malloc(), not a stack VLA like the earlier broken version. length here is coming from a cooperating process on the same machine rather than something adversarial, so this isn’t quite the same threat model as parsing untrusted input, but I still wouldn’t size a stack array off a number I read out of shared memory without thinking about it first. If you want the longer version of that argument, I wrote a whole piece on it: Safe Length-Based Data Sharing in C.

The pitfalls that get you anyway

POSIX named semaphores don’t go away when your process exits. They live in the kernel, backed by a file under /dev/shm on Linux, until something explicitly calls sem_unlink(), or the machine reboots. This bites people in a specific and annoying way: it can make the exact race this article is about disappear from your testing while still being sitting in your code.

Say you run the writer and reader once, cleanly. Writer posts, reader waits and consumes it, semaphore ends the run at 0. Now say you run the writer twice in a row, before ever running the reader, maybe because you’re testing something else entirely. Each run calls sem_post(). Since the semaphore already existed after the first run, the second run’s sem_open(SEM_NAME, O_CREAT, ...) just hands back the existing one, unchanged, because O_CREAT on an object that already exists ignores the initial-value argument you gave it. So after two writer runs, the count sits at 2.

Now the reader runs. Its sem_wait() sees a non-zero count and returns immediately, without ever actually blocking on anything. If you happen to be mid-refactor and there’s a bug that makes the next writer run crash before it writes anything, or skip the write entirely, you will not see it. The leftover count from an earlier run covers for it completely, and the reader sails through sem_wait() and reads whatever’s sitting in the segment from before. It’s the exact race from the top of this article, quietly back, except this time it’s hiding behind a semaphore that looks, from the outside, like it’s doing its job.

The fix is sem_unlink(SEM_NAME) and shm_unlink(SHM_NAME) between runs, so each run starts from a genuinely fresh semaphore at the value you meant, and a genuinely fresh, zeroed segment, instead of whatever the last run happened to leave behind.

The other thing worth deciding on purpose, not by accident: who owns O_CREAT. In the example above, the writer creates both the shared memory object and the semaphore, and the reader only ever opens what’s already there. That’s not arbitrary. If both sides pass O_CREAT, there’s no longer a clean answer to “who set the initial value,” because those creation arguments only take effect for whichever process’s open call happens to run first, and if both processes start around the same moment, that’s a race of its own. Pick one side to own creation, have it create both objects before doing anything else, and have the other side open them without O_CREAT, so a missing object fails loudly instead of getting created ambiguously by whichever process got there first.

So, back to that segfault

If you’re staring at a reader that segfaults sometimes, or hands back garbage sometimes, and every syscall you’re checking is returning success, it’s very likely this. Not a missing segment, not a bad permission bit, not a corrupted mapping. Just a reader that got to the memory before the writer was done with it, because nothing was stopping it from trying.

A semaphore initialized to 0, posted once by the writer after it’s actually finished, waited on once by the reader before it touches anything, closes that gap completely. Not by making the race less likely. By making it structurally impossible for the reader’s read to land before the writer’s write.

The problem

Here’s the kind of code that shows up when nobody has asked that question yet:

char buffer[256];
strcpy(buffer, source);

source is a pointer into the shared region. It looks like a string. It might even be a string, most of the time. So strcpy() gets called, and it works, in testing, on your machine, with the inputs you happened to try.

The problem is that strcpy() doesn’t know three things:

• how large the memory region behind source actually is
• how large buffer, the destination, actually is
• whether there is a null terminator anywhere inside the valid region at all

It doesn’t ask. It just starts reading byte by byte from source, copying each one into buffer, until it hits a '\0'. That’s the entire contract of strcpy(): keep going until you see a zero byte.

source:
+----------------------------------+
| H | e | l | l | o | ... | ???  |
+----------------------------------+
                              ^
                       where is '\0'?

If the writing process filled the shared region with exactly "Hello" and nothing else, and the region happens to end right after those five bytes, there is no '\0' inside the valid data. strcpy() doesn’t know that “valid data” ended, so it keeps reading whatever comes next: uninitialized memory, another process’s leftover bytes, or an unmapped page that segfaults on touch.

This isn’t a strcpy bug. strcpy() is doing exactly what it was designed to do. The bug is upstream of it: we handed it a pointer and implicitly assumed a length that was never actually communicated.

The key idea: stop asking where, start asking how much

The fix isn’t a smarter string function. It’s a different question. Instead of:

“Where is the '\0'?”

the receiving side should be asking:

“How many bytes are valid, starting from this pointer?”

That second question has an answer that doesn’t depend on scanning memory and hoping. It’s a number, and if the writer tells you that number up front, you never have to guess.

This means representing the data explicitly as a length followed by a payload, instead of a bare, unmarked stream of bytes:

+------------+----------------------+
| length     | payload              |
+------------+----------------------+
             <---- length bytes --->

This is usually called length-delimited or length-prefixed data, and it’s a genuinely old idea, older than C strings themselves. The boundary of the data isn’t a special byte value hidden somewhere in the payload. It’s a number, sitting right next to the data it describes.

Why this works better than null termination

Length-prefixing isn’t just a workaround for the shared-memory case. It’s a strictly more general representation, because it works for arbitrary binary data, not just text.

Think about what a null terminator actually assumes: that the byte value 0x00 never legitimately appears inside your data. That’s a reasonable assumption for ASCII text. It’s not a safe assumption for, say, a serialized struct, an image, an encrypted blob, or a network packet.

+----+----+----+----+----+
| 41 | 00 | FF | 42 | 00 |
+----+----+----+----+----+

The second byte here is 0x00. Treat this as a C string, and anything reading it with strlen() or strcpy() would stop after just one byte, 0x41, silently discarding the rest of the payload. That 0x00 isn’t a terminator. It’s just a byte that happens to be zero, sitting in the middle of a binary blob with four more bytes after it.

So '\0' can’t be a universal boundary marker for arbitrary data, because for binary data, every byte value including zero is a legitimate payload byte. Length, on the other hand, doesn’t care what values the bytes hold. It just says how many of them belong to you.

Introducing memcpy()

Once you have an explicit length, the right copying primitive changes too.

memcpy(destination, source, length);

memcpy() copies exactly length bytes. It doesn’t search for anything inside them, and it doesn’t care whether byte 40 happens to be zero. It just moves the number of bytes you told it to move, and stops. That makes it the natural fit for [length][payload] data, since the whole point of that representation was to make “how many bytes” an explicit, known value instead of something discovered by scanning.

But here’s the part that’s easy to skip past: memcpy() is not automatically safe just because you used it instead of strcpy().

char buffer[256];
memcpy(buffer, source, length);

If length is 1000, this overflows buffer just as badly as an unterminated strcpy() would. memcpy() trusts you completely. It will happily walk off the end of source, or write off the end of buffer, or both, if you hand it a length that doesn’t actually fit. Switching functions didn’t buy you safety. What buys you safety is validating the length before you ever call memcpy() at all.

The two boundaries you’re actually protecting

This is the part worth slowing down for, because it’s easy to only think about half of it.

Every copy has two separate regions of memory involved, and each one has its own size that the length has to respect.

The source boundary: how much valid data actually exists at the other end of that pointer.

+---------------------------+
| valid source region       |
+---------------------------+

The destination boundary: how much space you actually have to write into.

+----------------+
| destination    |
+----------------+

A copy is only safe when both of these hold at the same time:

length <= source_size
length <= destination_capacity

It’s tempting to check only one of these and feel done. Check only the source side, and you can overflow a destination buffer that was smaller than you assumed. Check only the destination side, and you can read past the end of a source region that was shorter than the length claimed. Both failures look identical from the outside: memory corruption, a crash, or worse, neither. The length has to fit both ends, not just one.

Validating the length against the region itself

Now bring this back to the shared-memory scenario we started with. Say the shared region is SIZE bytes total, and it’s laid out as a length field followed by the payload:

+----------------------+----------------------+
| length               | payload              |
+----------------------+----------------------+

Before touching the payload at all, the receiver has to check that the claimed payload actually fits inside the region it was given. The condition you want is:

if (length <= SIZE - sizeof(length)) {
    // safe to read `length` bytes of payload
}

It’s tempting to write this the other way around, checking sizeof(length) + length <= SIZE instead. Don’t. That version adds two numbers before comparing, and if length is attacker-controlled or corrupted, the addition can overflow and wrap around to a small number that sails right past the check while length itself is enormous. The subtraction form avoids this: SIZE - sizeof(length) is a fixed, known-safe value computed once, so you’re only ever comparing an untrusted number against it, never adding an untrusted number into something that could wrap.

Turning length-delimited data into a C string

Length-delimited data never needed a '\0' to begin with; the length told you exactly where it ends. But sometimes the code on the receiving end genuinely wants an ordinary C string, because it’s about to hand the data to a function that expects one.

That’s fine, as long as you understand it as a conversion step, not the native representation:

char string[length + 1];
memcpy(string, payload, length);
string[length] = '\0';

The + 1 and the manual assignment matter. The length-delimited payload is exactly length bytes, no more:

Length-delimited:
+----------------------+
| H e l l o            |
+----------------------+

A C string needs one extra byte tacked on at the end, the terminator, which was never part of the original data:

C string:
+-------------------------+
| H e l l o | \0         |
+-------------------------+

The shared representation on the wire, or in the shared region, stays length-based the whole time. The null terminator only gets added at the very last moment, locally, when you’re producing a C string for some function that specifically requires one. It’s a local convenience, not the source of truth for where the data ends.

Why char string[length] alone is still dangerous

Notice that the snippet above uses length + 1 on the stack. If length comes from shared or external data, you cannot trust it just because it’s sitting right there in a length field. A corrupted region, or one written by a process you don’t fully trust, can set length to whatever it wants, and an enormous value can blow the stack before you’ve written a single byte, trigger allocation failures your code never checked for, or drive reads and writes to addresses that have nothing to do with your actual data.

The length has to be validated against a real, known bound, like the region size we checked above, before it’s used to size an allocation or drive a copy. Once that check is done, it’s often simpler and safer to copy into a fixed-size buffer you already control, and reject or truncate anything that doesn’t fit, rather than sizing a new buffer dynamically off a number you just received.

A quick word on strncpy()

At this point someone always says: just use strncpy() instead of strcpy(), problem solved. It’s worth being clear about why that’s not the whole fix.

strncpy() takes a maximum number of bytes to copy, which sounds like what we want. But it still thinks in terms of strings: if the source runs out of bytes before hitting '\0', it pads the rest of the destination with zero bytes, and if the source doesn’t contain a '\0' within the given max length, it won’t add one for you either. You can walk away with an unterminated buffer, having done exactly what the function documented, and still be back at square one.

strncpy() is a safer string function. It is not a length-delimited-data function. It still frames the problem in terms of a maximum scan for a terminator, not an explicit number of valid bytes. The actual fix was never “pick a better string function.” It’s moving away from string-shaped thinking entirely, from an implicit length discovered by scanning to an explicit length that’s known up front.

The takeaway

In C, a pointer tells you where the bytes are. It does not tell you how many of them are valid.

That single sentence is the whole article, if you need to remember one thing from it. Everything else follows from taking it seriously:

• Pointer = where the data starts.
• Length = how much of it is actually yours to read.
• Buffer capacity = how much you can safely write, a separate number from the length.
• '\0' = a string terminator, useful for text, not a universal marker for where arbitrary data ends.
• memcpy() = copies exactly the number of bytes you tell it to, nothing more, nothing less, and trusts you completely to have gotten that number right.
• Validation = the actual step that makes any of this safe, checked against both the source region and the destination buffer, before a single byte moves.

None of this is exotic. It’s the same idea underneath length-prefixed protocol messages, struct-based binary formats, and most serious binary parsers you’ll come across. The pointer was never going to tell you how many bytes were valid. You have to bring that number with you.

Starting the container

docker run -it --rm --mount type=bind,source="$(pwd)"/work,target=/work -p 8888:8888 opencvcourses/opencv:440

What each flag does:

• -it starts an interactive shell. This switch is always needed to start the container, otherwise it will start and stop instantly.
• --rm specifies to kill the container after it is exited.
• --mount creates persistent storage to save all the work.
• -p exposes a container’s port to the host.

Note: run this command in the parent directory of the work folder.

Saving the state

You can save the state by committing the changes to the pulled image:

docker ps
docker commit <container_id> repo/testimage:version3

That’s it. Next time you run the image repo/testimage:version3, everything is exactly where you left it.

Is this good practice?

While this works, this approach is considered poor practice by the Docker community. The best way to go is using a Dockerfile.

And yeah, I think that’s the right way for production, for sure. But many of us are just using Docker locally as a slimmed down version of VirtualBox, and just want the damn state saved, exactly as it is. And we don’t want extra volumes on our local drives either.

But its installation can be very tricky in an environment like Linux, so let’s follow a correct installation process.

Download the sources

The most recent release at the time of writing is 4.5.1.

Create a tmp folder for all the archives:

mkdir ~/opencv4.5-tmp && cd ~/opencv4.5-tmp

Download the OpenCV sources and the opencv-contrib sources:

wget https://github.com/opencv/opencv/archive/4.5.1.zip -O opencv.zip
wget https://github.com/opencv/opencv_contrib/archive/4.5.1.zip -O opencv_contrib.zip

Unzip both archives:

unzip opencv.zip
unzip opencv_contrib.zip

Move the files to simpler directory names:

mv opencv-4.5.1/ opencv
mv opencv_contrib-4.5.1/ opencv_contrib

Build and install

Make a build directory:

cd opencv && mkdir build && cd build

Copy and run the following command (install cmake first if it is not available on your system):

cmake -D CMAKE_BUILD_TYPE=DEBUG \
      -D CMAKE_INSTALL_PREFIX=~/opencv4.5-custom \
      -D OPENCV_EXTRA_MODULES_PATH=~/opencv4.5-tmp/opencv_contrib/modules \
      -D OPENCV_GENERATE_PKGCONFIG=ON \
      -D BUILD_EXAMPLES=ON ..

Make the project:

make -j4

Install OpenCV:

sudo make install

Ensure it is updated in the library storage:

sudo ldconfig

Configure a C++ project to work with OpenCV

Open your editor of choice (vim in my case), create a folder ~/projects/HelloOpenCV, and put your code in main.cpp.

First, let’s try to compile the application with g++ as usual:

g++ -Wall -o main main.cpp

We see that it cannot find our library headers.

For that we need to provide the path to the headers and the linker flags. The best way to find them all in one place is the pkg-config utility. Remember we provided an additional argument to our cmake generation? So let’s execute the following:

export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/home/parallels/opencv4.5-custom/lib/pkgconfig
pkg-config --cflags --libs opencv4

Now add all the flags to the compilation command:

g++ -Wall -o main main.cpp $(pkg-config --cflags --libs opencv4)

Or a more explicit version for our particular sample:

g++ -Wall -o main main.cpp \
   -I/home/parallels/opencv4.5-custom/include/opencv4 \
   -L/home/parallels/opencv4.5-custom/lib \
   -lopencv_highgui -lopencv_videoio -lopencv_imgcodecs -lopencv_core

This same technique is used in storage management by cloud service providers like AWS and Azure.

Linux distribution: AlmaLinux 9.4 on VMware Workstation 7

When you expand the disk size of your Linux virtual machine on VMware, you must adjust the partitions within the system to utilize the additional space. This guide will walk you through the steps required to resize the partitions and filesystems using bash.

Prerequisites

Before you begin, ensure you have:

• Expanded the disk size of your VM in VMware.
• Root or sudo access to your Linux VM.

Step 1: Identify the new disk size

First, verify the current disk and partition sizes using the lsblk command:

lsblk

The output will show all block devices and their partitions. Identify the disk you have expanded. For example:

NAME               MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sr0                 11:0    1  988M  0 rom  
nvme0n1            259:0    0  100G  0 disk 
├─nvme0n1p1        259:1    0    1G  0 part /boot
└─nvme0n1p2        259:2    0   19G  0 part 
  ├─almalinux-root 253:0    0   17G  0 lvm  /
  └─almalinux-swap 253:1    0    2G  0 lvm  [SWAP]

Step 2: Resize the partition

Use the growpart utility to resize the partition. If it’s not already installed, you can install it with:

sudo dnf install cloud-utils-growpart

Then resize the partition nvme0n1p2:

sudo growpart /dev/nvme0n1 2

Step 3: Resize the physical volume (PV)

Next, resize the physical volume to recognize the expanded partition:

sudo pvresize /dev/nvme0n1p2

Step 4: Verify the physical volume size

Verify the changes using the pvs command:

sudo pvs

Step 5: Resize the logical volume (LV)

Assuming you want to allocate all the new space to the root logical volume (almalinux-root):

sudo lvextend -l +100%FREE /dev/almalinux/root

Step 6: Determine the filesystem type

Before resizing the filesystem, determine the filesystem type. You can do this with either of these commands:

df -Th

or

lsblk -f

Step 7: Resize the filesystem

Depending on the filesystem type, use the appropriate command.

For ext4:

sudo resize2fs /dev/almalinux/root

For xfs:

sudo xfs_growfs /

Step 8: Verify the changes

Finally, verify that the new space is available:

lsblk
df -h

Conclusion

By following these steps, you can effectively utilize the additional disk space after expanding your Linux VM disk on VMware, without losing any data or any running work.

Go see it yourself

If you open a compiled Java file, aka file.class, with a hex editor like the xxd tool in Linux, you’ll find the first hex words are:

CAFE BABE

But why CAFE BABE?

Why is it so magical that it has to be added to each class file? That’s probably the most important question to be asked.

Well, James Gosling explained why:

We used to go to lunch at a place called St Michael’s Alley. According to local legend, in the deep dark past, the Grateful Dead used to perform there before they made it big. It was a pretty funky place that was definitely a Grateful Dead Kinda Place. When Jerry died, they even put up a little Buddhist-esque shrine. When we used to go there, we referred to the place as Cafe Dead. Somewhere along the line, it was noticed that this was a HEX number. I was re-vamping some file format code and needed a couple of magic numbers: one for the persistent object file, and one for classes. I used CAFEDEAD for the object file format, and in grepping for 4 character hex words that fit after “CAFE” (it seemed to be a good theme) I hit on BABE and decided to use it. At that time, it didn’t seem terribly important or destined to go anywhere but the trash can of history. So CAFEBABE became the class file format, and CAFEDEAD was the persistent object format. But the persistent object facility went away, and along with it went the use of CAFEDEAD. It was eventually replaced by RMI.

So every Java class file in the world starts with the name of a cafe where the Grateful Dead used to play. If you know, you know.

So hello, let’s do it in C on an x86-64 bits machine. Why? Because it’s my favorite language.

From C to assembly

First, when you compile your C code:

gcc -c main main.c

you get an object file main.o that will be linked later (manually or dynamically) with other already compiled standard code (like your famous printf()), and you get an executable file, ELF64 in my case.

When you disassemble it, either with objdump or gdb, you can see the assembly code of your program:

gdb ./main
disassemble function

You’ll find some sections like .text, .bss or .data, the heap, and the stack. These are the sections of the virtual address space of your program.

What concerns us is the .text section: it contains the code, the instructions to be executed. For example int x = 3 will have the equivalent of movl $0x3,-0x4(%rbp).

Okay, good. What concerns us now is how the functions main() and function() are handled in the stack, the section where the functions reside.

The registers involved

The CPU has some special registers called %rbp and %rsp (the base pointer and the stack pointer) to manipulate the scope of the functions:

• %rbp is used to access the memory words by adding values to it, like the example we had: movl $0x3,-0x4(%rbp)
• %rsp always points to the top of the stack.

The function prologue

Here is the process, step by step:

1. The first thing we need to do is push the value of the old %rbp, because it’s used by other functions to do the same thing. In this case that’s the main() function, which called function().

2. Then we push the return address to the stack, so the main() function resumes its execution where it stopped (in this case it will return to line 13). This information is retrieved from register r14 in ARM processors, for example.

3. Then we’ll set the new value of the base pointer %rbp (the one we said is used for accessing memory words in the scope) to the value of %rsp, so the top of the stack.

This process of handling %rbp and %rsp is called the function prologue, and it’s done at each function start, as you can see in the assembly code.

This is how the initialization of the function scope is done, similarly across multiple architectures and systems.

Calling conventions

Now we’ll use the function call stack by putting the local variables, the function parameters, etc. These specific details follow calling conventions:

https://lnkd.in/eUNNAiXH

One of the conventions to follow on x64 architectures, for example, is System V. For function parameter handling, the registers are used like this: the first parameter is placed in rdi, the second in rsi, the third in rdx, and then rcx, r8 and r9. Only the 7th argument and onwards are passed on the stack. The left most parameter is passed first on the stack, then the old value of the base pointer, and then it’s followed by the local variables of the function.

Programs that tend to exhibit good locality are those constituted of components that tend to reference data items that are near other recently referenced data items, or that were recently referenced themselves.

The two types of locality

1. Temporal locality: if a main memory word is referenced once, and there are potential references in the near future, it should be in the cache memory.

2. Spatial locality: if a memory location is referenced once, then the program is likely to reference a nearby memory location in the near future. A good example is a programming block which is logically distinguishable, that is: traversing a contiguous array of data using a loop statement.

Okay, thank you for giving me information I can get from Wikipedia. How the heck will that help me improve my programming skills and performance?

Answer: we have the following example that performs a matrix values sum.

The sumarraycols() function in Figure 6.19(a) computes the same result as the sumarrayrows() function in Figure 6.18(a). The only difference is that we have interchanged the i and j loops.

But what impact does interchanging the loops have on locality?

The sumarraycols() function suffers from poor spatial locality because it scans the array column-wise instead of row-wise, since C arrays are laid out in memory row-wise.

The examples are retrieved from the book: Computer Systems - A Programmer’s Perspective.

Let’s discover that together, step by step:

Figure 2.11 - Book reference: Operating System Concepts.

From binary on disk to a running process

Usually programs, a.out or program.exe (games, word processors, web browsers…), reside on your disk as binary executable files. In order to execute one of them, you need to fetch it from disk and put it in memory, where it waits until it’s scheduled to run. It then becomes a running program, also known as a “process”, which has a specific amount of memory addresses and is executed by the units of the CPU’s core with the help of specific registers like program counters, IR, and general-purpose registers.

But we don’t write operating systems and enterprise systems in binary, right?

We need a human readable language. Let’s choose a compiled programming language like C to understand the process of compilation.

Compiling and assembling

After writing your program.c source code in human readable C, the source code is compiled and assembled into a relocatable object file program.o: machine code that can be disassembled and turned into geek-readable assembly. (We will talk about the phases of compilation, lexing, parsing, AST… in future articles.)

So we have now an object file. But we may have called several functions like printf(), which were also written by system programmers before us, to give us a layer of abstraction and increase our productivity. They are compiled, aka turned into an object file too, as part of GCC.

Linking

We now need to link all those object files and make them a single binary executable file that can be loaded into memory. This mechanism is done by what we call a “linker”. (This is static linking.)

Loading

The mechanism of loading the binary executable file into memory, so it can be executed by one of the CPU’s cores, is done by the loader. It’s also responsible for address reallocation in memory.

Dynamic linking

You may have heard about DLL (dynamically linked libraries) files in Windows, right? Now we speak about dynamic linking. Actually, most systems allow programs to dynamically link libraries even if the program is already loaded and started execution. That’s good from a memory optimization perspective, because the programmer may include a library and not use it during runtime. I am sure we all did that before.

And you know that modern systems allow multiple processes to share a DLL? Yes, they do!

And now you have a binary executable program in memory, also known as a process, with its own memory addresses.

I will talk deeper next time about the compilation phases, and how the CPU fetches, decodes, and executes instructions and data from the bounded virtual memory of the process.

In the Linux kernel, specifically in the PCB (process control block), there is a structure that represents the process in the kernel: task_struct.

The six state flags

The process state is represented by a combination of six state flags:

• TASK_RUNNING: the process is currently running or ready to run.
• TASK_INTERRUPTIBLE: the process is waiting for a specific event to occur and can be interrupted by a signal.
• TASK_UNINTERRUPTIBLE: the process is waiting for a specific event to occur and cannot be interrupted by a signal.
• __TASK_STOPPED: the process has been stopped (e.g. by a SIGSTOP signal) and can be resumed later.
• __TASK_TRACED: the process is being traced by another process (e.g. a debugger).
• TASK_DEAD: the process has terminated and is waiting to be reaped by its parent process.

Combining flags

Each of these flags can be set or cleared to represent the different states a process can be in. For example, a process that is both TASK_INTERRUPTIBLE and TASK_UNINTERRUPTIBLE might be waiting for a disk I/O operation to complete, and can be interrupted by a signal but cannot be killed until the I/O operation is finished.

In other words, a process can have multiple states at once, depending on which combination of flags is set.

So why 4 bytes?

This state is saved in an unsigned integer. That means 32 bits, which means 2³² possible values. But we only need 2⁶, so 64 possible states as a maximum.

The reason is the following:

There are some predefined functions that manage the process’s state, such as READ_ONCE/WRITE_ONCE, that oblige this variable to be an unsigned int. It used to be a volatile long. Here is the commit:

https://github.com/torvalds/linux/commit/2f064a59a11ff9bc22e52e9678bc601404c7cb34#diff-f8d8a1568ae83bbff6f40f9c70559a4f7dbf426a397131ba9d4fbfb947ea5222R669

Thank you for reading <3