gp_overflow(5)
gp_overflow - GP Overflow Errors
As shipped in IRIX 6.5. First release of IRIX 6.5.
NAME gp_overflow - GP Overflow Errors TOPIC This man page is intended to describe the underlying causes of and possible solutions for overflowing the gp-relative area in the linker. The Symptoms and a Simple Solution GP overflow is reported by the linker with one of several error messages: GP-relative sections overflow by 0x??? bytes. Please recompile with a smaller -G value. GOT overflow: please relink with -multigot. GOT unreachable: please relink with -multigot. These messages all indicate that the code being linked contains more than 64KB of GP-relative data area (see the next section), and attempts to reference it with a signed 16-bit displacement from the GP register. In the past we recommended recompiling with -xgot. Although this generally solved the GP overflow problem it created larger, slower programs. We developed -multigot to create multiple GOTs during linking: -multigot prevents the GOT overflow problem while retaining the more efficient GP relative code. -xgot will be dropped in future releases. The preferred solution to the problem is to use the -multigot option to ld. At some point -multigot is likely to become the default default. Even with -multigot one must still avoid having too much GP data: if the -G value is greater than zero it may be necessary to use a smaller -G value (see below for definitions). Terminology GP GP (sometimes spelled gp) is a register in the register set of the processor. It used in a consistent way by the software. The register is actually an ordinary integer register (register 28) but by software convention it is used only in a stylized way. The GP- area is a region of memory addressable by an offset of the GP register. See the section "Underlying Mechanisms" below. GOT To provide Position Independent Code (PIC) which can be relocated without modifying the text of the program (or DSO) it is sufficient to have a data table with the actual addresses of global data (with appropriate code generation and linking support). This data table is constructed by ld(1) and is called the Global Offset Table (GOT). For a further explanation of what the GOT is, see dso(5). PIC Position Independent Code (PIC) is, as mentioned above, a method of code generation which results in code that can be shared by multiple users (each program must have its own data space! Code sharing and independent data is arranged automatically by the compilation and run-time systems). For details on PIC code generation, see the dso(5) (for old-32bit-abi) or the "MIPSpro N32 ABI Handbook" (for the new-32bit-abi). -G n This compiler option sets the -G value, n. This compiler option affects how much data goes into the "small data" and "small bss" and other areas (discussed in the "Underlying Mechanisms" section below where short data segment is defined) ). The default n for old- 32bit-abi compilation (when producing PIC code) is 0. The default for new-32bit-abi and 64bit-abi compilation is 8. To see the value actually in use, use the -show option on the compilation line and check the -G option actually passed to the compiler passes. Underlying Mechanisms The MIPS instruction set architecture provides instructions for loading values from memory, and storing values to memory, which calculate an effective address by adding the content of a register to a signed 16-bit immediate value (displacement) in the instruction. The compilers take advantage of this capability by maintaining various frequently-used data objects in a number of small memory sections, keeping a pointer to this area of memory in a dedicated register (GP), and generating references to those objects as simple load/store instructions with signed 16-bit displacements If the total size of these memory sections exceeds 64K bytes, the displacements in some of those instructions will likely overflow, and the linker reports a GP overflow error. We will refer to these small GP-relative memory sections collectively as the short data segment below. In general, it is comprised of the Unix sections .sdata, .sbss, The data objects placed in the short data segment by the compiler and linker include: Static data page pointers: These are no longer a problem with current code generation and linkers. GOT pointers: References to static data or subroutine addresses in shared code, except for those which use page pointers mentioned above, normally load an address from the GOT, relative to GP, and use it for the access. By default, these address loads assume an offset from GP of 32KB or less. Using the -multigot linker option tells the linker to arrange for as many GOTs as needed in the a.out or DSO (so the fact each GOT is limited in size becomes irrelevant). Using the (now deprecated) -xgot compiler option (or -TENV:large_GOT option when compiling n32 or 64-bit programs using the MIPSPro compiler) forces the compiler to assume full 32-bit offsets from GP, and thereby eliminates the restriction that these pointers fit in 64KB. Small data objects: The compiler attempts to place small data objects in a GP-relative section (e.g. .sdata or .sbss or .lit4 or .lit8) and to reference them directly using a signed 16-bit displacement. The limit on the size of objects treated this way can be changed from the default to n bytes by using the -G n option These options must be used consistently for all compilations in a given program or linker errors will occur. You might get one of these errors: "foo" used in bar.o and baz.o has different sizes. "foo" used as gp-relative in bar.o and defined otherwise in baz.o. "foo" cannot be turned into gp-relative. The data objects affected by -G n are numeric literals, addresses including those generated by the compiler, all C static variables, and (if the "-static"flag was used) FORTRAN local variables. Small literal objects: The compiler also attempts to place small literal objects (e.g. integer constants, floating point constants, addresses) in a GP- relative section (e.g. .lit4, .lit8, or .srdata) and to reference them directly using a signed 16-bit displacement. The limit on the size of literal objects treated this way can be changed from the default to n bytes by using the -G n option. The -G option must be used consistently for all compilations in a given program, or linker errors are likely to occur. Each of these uses normally has a positive effect on program performance. (In fact, performance can often be improved further by increasing the sizes of data and literal objects allocated to the short sections.) However, if the total size of the four categories exceeds 64KB, the linker will report GP overflow errors, and the program must be rebuilt to shrink the total size of GP-relative data. Tailoring a Solution for Best Performance The simplest solution is to use the -multigot option. Finding the best solution requires identifying the sizes of the short data section components (page pointers, GOT pointers, small data, and small literals), and removing only as much as necessary. With this release this information may be obtained from the linker by specifying the flag "- gpinfo" directly to ld(1), or by specifying "-Wl,-gpinfo" to cc(1) or f77(1). In other cases, this information may be obtained by specifying the "-m -aoutkeep" flags to ld(1) (or -Wl,-m,-aoutkeep to cc(1) or CC(1) or f77(1)) and subsequently searching the load map for the desired sections. Each of these components can be controlled: Page pointers: They are no longer an issue: compiler and linker changes now make the the -TENV:no_page_offset option unnecessary. GOT pointers: These must be GP-relative, but compiling with -multigot enables the use of multiple gp regions and the linker will place the short data and literal sections closer to an appropriate GP so that they can still benefit. Small data: Compiling with -G n for n smaller than the existing value shrinks these sections. Use -G 0 to eliminate all GP-relative small data references. (Note that some small data references will be replaced by new GOT pointer references, so the savings might not be as large as the original size of the short data sections.) Recompiling with this option requires recompiling the entire program with it. Small literals: Compiling with -G n for n smaller, shrinks these sections. Use -G 0 to eliminate all GP-relative small literal references. (Note that some small literal references will be replaced by new GOT pointer references, so the savings might not be as large as the original size of the short literal sections.) Recompiling with -G n requires recompiling the entire program with it. There is another option which may perform better than the above. The main program and each DSO it uses each get their own short data segments. Breaking parts of the program into separate DSOs may shrink all of the short data segment components enough that the defaults work for some or all of the pieces. However, be aware that subprogram calls which cross DSO boundaries incur some extra overhead, so this solution is likely to help performance only if the division can be made such that such crossings are infrequent. The -xgot option The following is retained to help make clear what the options used with -xgot were so you can remove them and use -multigot instead. Please do not use -xgot. Use -multigot. -multigot requires no recompiling, just relinking. (Any program compiled -xgot should be completely recompiled to use -multigot.) In the future -multigot will be on by default. The -xgot rules suggested for the earlier compiler releases were as follows: For n32 or 64-bit programs, recompile everything in one's program with the options: -avoid_gp_overflow this eliminates the problem. This flag is a synonym for -G 0 -xgot -TENV:no_page_offset Programs compiled for the old 32bit abi recompile with -G 0 -xgot -xgot is likely to have a significant negative impact on the program's performance. BUGS ld earlier than IRIX 6.2 produced one page pointer per 64KB page of static data, whether or not that page is ever referenced. As a result, 256MB of static data would fill the GP-relative area with page pointers, leaving no space for the other GP-relative memory sections. More than that would normally cause linker errors unless the -TENV:no_page_offset option was used for compilation. The 6.2 and later linkers are much more efficient in its data GOT page production making the option -TENV:no_page_offset unnecessary. A workaround for this problem was to compile all large data blocks from files which contain no programs of any kind, and link them into a separate data-only dso. Since it contains no code, there will be no references to the overflowed GOT. This dso can then be linked against when building the main program. Since references to it are always via the name of the symbol (or the name of the common in FORTRAN), the data will be referenced correctly, but since the data is defined in a dso, space will not be allocated in the main program. For example, suppose we wanted to define an array of 1,000,000,000 floating point values. We could then write the following source file in C: float bigarray[1000000000]; And subsequently compile it and link it thus: cc -c bigarray.c ld -shared bigarray.o -o bigarray.so Then we would link the main program thus: cc <objects> bigarray.so <libraries> To run it we must either install bigarray.so into /usr/lib, or else tell the runtime linker rld(1) where to find it. One way of doing this is setting the environment variable LD_LIBRARY_PATH to the pathname of the directory in which bigarray.so can be found. FORTRAN programmers use the source file BLOCKDATA DUMMY COMMON/mycom/bigarray(1000000000) END Compile it with f77(1), rather than with cc(1), the other steps are the same. SEE ALSO cc(1), f77(1), ld(1), dso(5), MIPSpro N32 ABI Handbook