I am coding a simple Unicode character library in C based off of the rune API found in the plan9 libc. This is primarily a project for me to learn about Unicode but I will be using the library in my future projects. I have found the most complex part of the project is dealing with the character data. My rough draft implementation was use the fact that the Unicode data was in sequential code point order and extract the beginning and end code point for a series of related character types and put that all in a table.
However as I was reading the Unicode spec I came across a section on implementation in Chapter 5 section 1.2 I came across this chunk of text.
Ranges. It may be tempting to “optimize” these tables for space by providing elaborate provisions for nested ranges or similar devices. This practice leads to unnecessary performance costs on modern, highly pipelined processor architectures because of branch penalties. A faster solution is to use an optimized two-stage table, which can be coded without any test or branch instructions. Hash tables can also be used for space optimization, although they are not as fast as multistage tables.
The spec essentially says my original idea is a mistake and to either implement and it is more efficient to either make a flat lookup table or a multi staged lookup table. I had considered making just a big table a bit naive but it would be much simpler than my current implementation.
Would having a large .rodata section in the library have any downside on modern CPU?