Learn a new trick with the offsetof() macro

http://www.embedded.com/shared/printableArticle.jhtml?articleID=18312031

Learn a new trick with the offsetof() macro
By Nigel Jones, Courtesy of Embedded Systems Programming
3?11 2004 (15:00 $?
URL: http://www.embedded.com/showArticle.jhtml?articleID=18312031

Click here for reader response to this article

Almost never used, the offsetof() macro can actually be a helpful addition to your bag of tricks. Here are a couple of places in embedded systems where the macro is indispensable—packing data structures and describing how EEPROM data are stored.

If you browse through an ANSI C compiler's header files, you'll come across a very strange looking macro in stddef.h. The macro, offsetof(), has a horrid declaration. Furthermore, if you consult the compiler manuals, you'll find an unhelpful explanation that reads something like this:

"The offsetof() macro returns the offset of the element name within the struct or union composite. This provides a portable method to determine the offset."

At this point, your eyes start to glaze over, and you move on to something that's more understandable and useful. Indeed, this was my position until about a year ago when the macro's usefulness finally dawned on me. I now kick myself for not realizing the benefits earlier—the macro could have saved me a lot of grief over the years. However, I console myself by realizing that I wasn't alone, since I'd never seen this macro used in any embedded code. Offline and online searches confirmed that offsetof() is essentially not used. I even found compilers that had not bothered to define it. How the macro works

Before delving into the three areas where I've found the macro useful, it's necessary to discuss what the macro does, and how it does it.

The offsetof() macro is an ANSI-required macro that should be found in stddef.h. Simply put, the offsetof() macro returns the number of bytes of offset before a particular element of a struct or union.

The declaration of the macro varies from vendor to vendor and depends upon the processor architecture. Browsing through the compilers on my computer, I found the example declarations shown in Listing 1. As you can see, the definition of the macro can get complicated.

Listing 1: A representative set of offsetof() macro definitions

// Keil 8051 compiler
#define offsetof(s,m) (size_t)&(((s *)0)->m)

// Microsoft x86 compiler (version 7)
#define offsetof(s,m) (size_t)(unsigned long)&(((s *)0)->m)

// Diab Coldfire compiler
#define offsetof(s,memb) ((size_t)((char *)&((s *)0)->memb-(char *)0))

Regardless of the implementation, the offsetof() macro takes two parameters. The first parameter is the structure name; the second, the name of the structure element. (I apologize for using a term as vague as "structure name." I'll refine this shortly.) A straightforward use of the macro is shown in Listing 2.

Listing 2: A straightforward use of offsetof()

typedef struct
{

	int	i;
	float	f;
	char	c;

} SFOO;

void main(void)
{
  printf("Offset of 'f' is %u", offsetof(SFOO, f));
}

To better understand the magic of the offsetof() macro, consider the details of Keil's definition. The various operators within the macro are evaluated in an order such that the following steps are performed:

1. ((s *)0) takes the integer zero and casts it as a pointer to s.
2. ((s *)0)->m dereferences that pointer to point to structure member m.
3. &(((s *)0)->m) computes the address of m.
4. (size_t)&(((s *)0)->m) casts the result to an appropriate data type.

By definition, the structure itself resides at address 0. It follows that the address of the field pointed to (Step 3 above) must be the offset, in bytes, from the start of the structure. At this point, we can make several observations:

• We can be a bit more specific about the term "structure name." In a nutshell, if the structure name you use, call it s, results in a valid C expression when written as (s *)0->m, you can use s in the offsetof() macro. The examples shown in Listings 3 and 4 will help clarify that point.
• The member expression, m, can be of arbitrary complexity. Indeed, if you have nested structures, then the member field can be an expression that resolves to a parameter deeply nested within a structure.
• It's easy enough to see why this macro also works with unions.
• The macro won't work with bitfields. You simply can't take the address of a bitfield member of a structure or union.

Listings 3 and 4 contain simple variations on the usage of this macro. These should help you get you comfortable with the offsetof() basics.

Listing 3: Without a typedef

struct sfoo {

	int	i;
	float	f;
	char	c;

};

void main(void)
{
  printf("Offset of 'f' is %u", offsetof(struct sfoo, f));
}

Listing 4: Nested structs

typedef struct
{

	long	l;
	short	s;

} SBAR;

typedef struct
{

	int	i;
	float	f;
	SBAR	b;

} SFOO;

void main(void)
{
  printf("Offset of 'l' is %u",     offsetof(SFOO, b.l));
}

Now that you understand the semantics of the macro, it's time to take a look at how to use it.

Use 1: pad bytes
Most 16-bit and larger processors require that data structures in memory be aligned on a multibyte (for example, 16-bit or 32-bit) boundary. Sometimes the requirement is absolute, and sometimes it's merely recommended for optimal bus throughput. In the latter case, the flexibility is offered because the designers recognized that you may wish to trade off memory access time with other competing issues such as memory size and the ability to transfer (perhaps via a communications link or direct memory access) the memory contents directly to another processor that has a different alignment requirement.

For cases such as these, it's often necessary to resort to compiler directives to achieve the required level of packing. As the C structure declarations can be quite complex, working out how to achieve this can be daunting. Furthermore, after poring over the compiler manuals, I'm always left with a slight sense of unease about whether I've really achieved what I set out to do.

The most straightforward solution to this problem is to write a small piece of test code. For instance, consider the moderately complex declaration given in Listing 5.

Listing 5: A union containing a struct

typedef union
{

	int	i;
	float	f;
	char	c;

struct {

	float	g;
	double	h;

  } b;
} UFOO;

void main(void)
{
  printf("Offset of 'h' is %u",     offsetof(UFOO, b.h)); }

If you need to know where b.h resides in the structure, then the simplest way to find out is to write some test code such as that shown in Listing 5. This is all well and good, but what about portability? Writing code that relies on offsets into structures can be risky—particularly if the code gets ported to a new target at a later date. Adding a comment is of course a good idea—but what one really needs is a means of forcing the compiler to tell you if the critical members of a structure are in the wrong place. Fortunately, one can do this using the offsetof macro and the technique in Listing 6, courtesy of Michael Barr.

Listing 6: Anonymous union that checks structure offsets

typedef union
{

	int	i;
	float	f;
	char	c;
	struct
	{
	float	g;
	double	h;
	} b;

} UFOO;

static union
{
        char wrong_offset_i[offsetof(UFOO,i) == 0];
        char wrong_offset_f[offsetof(UFOO,f) == 0];

        char wrong_offset_h[offsetof(UFOO, b.h) == 2]; //Error generated
};

The technique works by attempting to declare an array. If the test evaluates to FALSE, then the array will be of zero size, and a compiler error will result. In Listing 6, the compiler generates an error "Invalid dimension size [0]" for the parameter b.h.

Thus the offsetof() macro allows you to both determine and check the packing of elements within structures.

Use 2: nonvolatile memory
Many embedded systems contain some form of nonvolatile memory, which holds configuration parameters and other device-specific information. One of the most common types of nonvolatile memory is serial EEPROM. Normally, such memories are byte addressable. The result is often a serial EEPROM driver that provides an API that includes a read function that looks like this:

ee_rd(uint16_t offset, uint16_t nBytes, uint8_t * dest)

In other words, read nBytes from offset offset in the EEPROM and store them at dest. The problem is knowing what offset in EEPROM to read from and how many bytes to read (in other words, the underlying size of the variable being read). The most common solution to this problem is to declare a data structure that represents the EEPROM and then declare a pointer to that struct and assign it to address 0x0000000. This technique is shown in Listing 7.

Listing 7: Accessing parameters in serial EEPROM via a pointer

typedef struct
{

	int	i;
	float	f;
	char	c;

} EEPROM;

EEPROM * const pEE = 0x0000000;

ee_rd(&(pEE->f), sizeof(pEE->f), dest);

This technique has been in use for years. However, I dislike it precisely because it does create an actual pointer to a variable that supposedly resides at address 0. In my experience, this can create a number of problems including:

1. Someone maintaining the code can get confused into thinking that the EEPROM data structure really does exist.
2. You can write perfectly legal code (for example, pEE->f = 3.2) and get no compiler warnings that what you're doing is disastrous.
3. The code doesn't describe the underlying hardware well.

A far better approach is to use the offsetof() macro. An example is shown in Listing 8

Listing 8: Using offsetof()to access parameters in serial EEPROM

typedef struct
{

	int	i;
	float	f;
	char	c;

} EEPROM;

ee_rd(offsetof(EEPROM,f), 4, dest);

However, there's still a bit of a problem. The size of the parameter has been entered manually (in this case "4"). It would be a lot better if we could have the compiler work out the size of the parameter as well. No problem, you say, just use the sizeof() operator. However, the sizeof() operator doesn't work the way we would like it to. That is, we cannot write sizeof(EEPROM.f) because EEPROM is a data type and not a variable.

Now normally, one always has at least one instance of a data type so that this is not a problem. In our case, the data type EEPROM is nothing more than a template for describing how data are stored in the serial EEPROM. So, how can we use the sizeof() operator in this case? Well, we can simply use the same technique used to define the offsetof() macro. Consider the definition:

#define SIZEOF(s,m) ((size_t) sizeof(((s *)0)->m))

This looks a lot like the offsetof() definitions in Listing 1. We take the value 0 and cast it to "pointer to s." This gives us a variable to point to. We then point to the member we want and apply the regular sizeof() operator. The net result is that we can get the size of any member of a typedef without having to actually declare a variable of that data type.

Thus, we can now refine our read from the serial EEPROM as follows:

ee_rd(offsetof(EEPROM, f), SIZEOF(EEPROM, f), &dest);

At this point, we're using two macros in the function call, with both macros taking the same two parameters. This leads to an obvious refinement that cuts down on typing and errors:

#define EE_RD(M,D)   ee_rd(offsetof(EEPROM,M), SIZEOF(EEPROM,M), D)

Now our call to the EEPROM driver becomes much more intuitive:

EE_RD(f, &dest);

That is, read f from the EEPROM and store its contents at location dest. The location and size of the parameter is handled automatically by the compiler, resulting in a clean, robust interface.

Use 3: protecting nonvolatile memory
The last example contains elements from the first two examples. Many embedded systems contain directly addressable nonvolatile memory, such as battery-backed SRAM. It's usually important to detect if the contents of this memory have been corrupted. I usually group the data into a structure, compute a CRC (cyclic redundancy code) over that structure, and append it to the data structure. Thus, I often end up with something like this:

struct nv
{

	short	param_1;
	float	param_2;
	char	param_3;
	uint16_t	crc;

} nvram;

The intent of the CRC is that it be the CRC of "all the parameters in the data structure with the exception of itself." This seems reasonable enough. Thus, the question is, how does one compute the CRC? If we assume we have a function, crc16( ), that computes the CRC-16 over an array of bytes, then we might be tempted to use the following:

nvram.crc = crc16((char *) &nvram, sizeof(nvram)-sizeof(nvram.crc));

This code will only definitely work with compilers that pack all data on byte boundaries. For compilers that don't do this, the code will almost certainly fail. To see that this is the case, let's look at this example structure for a compiler that aligns everything on a 32-bit boundary. The effective structure could look like that in Listing 9.

Listing 9: An example structure for a compiler that aligns everything on a 32-bit boundary

struct nv
{

	short	param_1;	//Offset = 0
	char	pad1[2];	//Two byte pad
	float	param_2;	//Offset = 4
	char	param_3;	//Offset = 8
	char	pad2[3];	//Three byte pad
	uint16_t	crc;	//Offset = 12
	char	pad3[2];	//Two byte pad

} nvram;

The first two pads are expected. However, why is the compiler adding two bytes onto the end of the structure? It does this because it has to handle the case when you declare an array of such structures. Arrays are required to be contiguous in memory, too. So to meet this requirement and to maintain alignment, the compiler pads the structure out as shown.

On this basis, we can see that the sizeof(nvram) is 16 bytes. Now our nave code in Listing 9 computes the CRC over sizeof(nvram) - sizeof(nvram.crc) bytes = 16 - 2 = 14 bytes. Thus the stored CRC now includes its previous value in its computation! We certainly haven't achieved what we set out to do.

Listing 10: Nested data structures

struct nv
{
    struct data
    {

	short	param_1;	//Offset = 0
	float	param_2;	//Offset = 4
	char	param_3;	//Offset = 8

    } data;

uint 16_t

crc;

//Offset = 12

} nvram;

The most common solution to this problem is to nest data structures as shown in Listing 10. Now we can compute the CRC using:

nvram.crc = crc16((uint8_t *) &nvram.data, sizeof(nvram.data));

This works well and is indeed the technique I've used over the years. However, it introduces an extra level of nesting within the structure—purely to overcome an artifact of the compiler. Another alternative is to place the CRC at the top of the structure. This overcomes most of the problems but feels unnatural to many people. On the basis that structures should always reflect the underlying system, a technique that doesn't rely on artifice is preferable—and that technique is to use the offsetof() macro.

Using the offsetof() macro, one can simply use the following (assuming the original structure definition):

nvram.crc = crc16((uint8_t *) &nvram, offsetof(struct nv, crc));

Keep looking
I've provided a few examples where the offsetof() macro can improve your code. I'm sure that I'll find further uses over the next few years. If you've found additional uses for the macro I would be interested to hear about them.

Nigel Jones is a consultant living in Maryland, where he slaves away on a wide range of embedded projects. He enjoys hearing from readers and can be reached at najones@rmbconsulting.us.

---

---

Reader Response

Well whadya know! In my last large firmware project, I rolled my own offsetof() macro that looked like this:

//This macro is for hiding the tortured expression needed to extract
//the offset of the elements of the PAGE_TRAILER structure:
#define OFFSET_OF(element) ((char*)&(((PAGE_TRAILER*)0)->element) - (char*)0)

Had no idea it was provided in the standard library!

The IAR compiler which I use implements it like this:

#define offsetof(type,member) ((size_t)(&((type *)0)->member))

Even though it is in the STDDEF.H header file, it's not mentioned in the IAR documentation. Maybe that's why I missed it!

Thanks for pointing it out!

Brad Peeters
President
Theta Engineering

---

I enjoyed your article and would like to mention another use of the offsetof() macro. In the past I've found the offsetof() macro quite useful in providing an abstract interface, without having to use an object orientated high-level language (e.g. C++). For example, in the past I created a library function to calculate the standard deviation of a set of floats, through the use of a function with the following prototype:

  float stdDev(void *start,
    size_t stride,
    size_t offset,
    size_t num);

The value of start is typically the starting address of a static array of structures, that contains collected data. The sizeof of array entry is provided by the stride parameter. While how many there are is given by num. The value of num is typically obtained through the use of:

  #define NUM(_a) (sizeof(_a)
    / sizeof(_a[0]))

While the offset of the float type in the structure is given by the offset parameter. The nice thing about this function is that it has no idea of the structure type that it is dealing with. All it knows is that some specified number of bytes from the start of the structure is a value of type float.

I've also used similar techniques where the data is contained on a linked list or binary tree. If the data is on a linked list, instead of providing the stride and number of entries, the offset to the next pointer member can be provided. A common practice is to always have the next pointer at the begining of the structure. In this case a value of 0 is implied for the offset to the pointer to the next node. Sometimes the data I deal with is linked on multiple lists. When there are multiple lists, there will be multiple next pointers, usually with similar but always different member names. A function that takes the offset of the next pointer is very useful, in that it can work with structures that are linked onto multiple lists.

I made quick reference to also using this technique on binary trees. In this case the offset of the parent, left child, and right child nodes is needed. It tends to be awkward to pass the offset of these 3 pointers, plus the offset of the data to be processed as parameters. A solution to this issue is to maintain a structure that describes the entire tree. When the tree is created, a structure is created that contains the following:

  struct {
    struct treeAttributes {
      size_t parentOffset;
      size_t leftChildOffset;
      size_t rightChildOffset;
      size_t nodeSize;
      struct *root;
    };
  }

For this technique, it is agreed upon all the trees that use the generic functions, to always have a treeAttribute structure as the first member. With this, it's now possible to create functions such as:

  float stddevTree(void *tree,
    size_t floatOffset);

In this case, the tree parameter points to a structure of an unknown type, but it's agreed that its first member will be of type treeAttribute.

In the example, treeAttribute structure, I've also provided the size of each node. This wouldn't be needed to calculate the standard deviation, but is quite useful with other generic functions. For example, it can be used to automatically create duplicates of nodes with a specified attribute. These nodes might be written to a caller specified file descriptor, as a means to write selected nodes out to a file.

Louis Huemiller

Copyright 2003 © CMP Media LLC