WG14 decided to use the name C23 as the informal name of the next
revision of the C standard (notwithstanding the publication date in
2024). Update references to C2X in glibc to use the C23 name.
This is intended to update everything *except* where it involves
renaming files (the changes involving renaming tests are intended to
be done separately). In the case of the _ISOC2X_SOURCE feature test
macro - the only user-visible interface involved - support for that
macro is kept for backwards compatibility, while adding
_ISOC23_SOURCE.
Tested for x86_64.
The multibyte character needs to fit into the remaining buffer space,
not the already-written buffer space. Without the fix, we were never
moving the write pointer from the start of the buffer, always using
the single-character fallback buffer.
Fixes commit 04b76b5aa8 ("Don't error out writing
a multibyte character to an unbuffered stream (bug 17522)").
During the review of a GCC analyzer test case, we found most stdio
functions accepting a FILE * argument expect it to be nonnull and just
segfault when the argument is NULL. Add nonnull attribute for them.
fflush and fflush_unlocked are well defined when __stream is NULL so
they are not touched.
For fputs, fgets, fread, fwrite, fprintf, vfprintf, and their unlocked
version, if __stream is empty but there is nothing to read or write,
they did not segfault. But the standard disallow __stream to be empty
here, so nonnull attribute is also added for them. Note that this may
blow up some old code already subtly broken.
Also add __nonnull for _chk variants and __fortify_function versions for
them.
Signed-off-by: Xi Ruoyao <xry111@xry111.site>
Reviewed-by: Alejandro Colomar <alx@kernel.org>
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
IO_VTABLES_LEN is the size of the struct array in bytes, not the number
of __IO_jump_t's in the array. Drops just under 384kb from .rodata on
LP64 machines.
Fixes: 3020f72618 ("libio: Remove the usage of __libc_IO_vtables")
Signed-off-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Tested-by: Florian Weimer <fweimer@redhat.com>
On GNU/Hurd, O_RDWR actually is O_WRONLY|O_RDONLY, so checking through
bitness really is wrong. O_ACCMODE is there for this.
Fixes: 5324d25842 ("fileops: Don't process ,ccs= as individual mode flags (BZ#18906)")
In processing the first 7 individual characters of the mode for fopen
if ,ccs= is used those characters will be processed as well. Stop
processing individual mode flags once a comma is encountered. This has
the effect of requiring ,ccs= to be the last mode flag in the mode
string. Add a testcase to check that the ,ccs= mode flag is not
processed as individual mode flags.
Reviewed-by: DJ Delorie <dj@redhat.com>
The change is meant to avoid unwanted PLT entry for the fgets_unlocked
routine when _FORTIFY_SOURCE is set.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Move declarations from libio/bits/stdio.h to existing
libio/bits/stdio2-decl.h. This will enable future use of
__REDIRECT_FORTIFY in place of some __REDIRECT.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Since the _FORTIFY_SOURCE feature uses some routines of Glibc, they need to
be excluded from the fortification.
On top of that:
- some tests explicitly verify that some level of fortification works
appropriately, we therefore shouldn't modify the level set for them.
- some objects need to be build with optimization disabled, which
prevents _FORTIFY_SOURCE to be used for them.
Assembler files that implement architecture specific versions of the
fortified routines were not excluded from _FORTIFY_SOURCE as there is no
C header included that would impact their behavior.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Now that abort no longer calls fflush there is no reason to avoid locking
the stdio streams anywhere. This fixes a conformance issue and potential
heap corruption during exit.
With fortification enabled, system calls return result needs to be checked,
has it gets the __wur macro enabled.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
With fortification enabled, fread calls return result needs to be checked,
has it gets the __wur macro enabled.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Calling fclose or freopen with a null FILE * is undefined behavior, and
doing so in practice will cause a SIGSEGV. So it seems suitable for
__nonnull.
This will help the compiler to warn for some buggy code, like
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109570.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
GCC docs explicitly list perror () as a good candidate for using
__attribute__ ((cold)). So apply __COLD to perror () and similar
functions.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
Message-Id: <20230429131223.2507236-3-bugaevc@gmail.com>
FreeBSD makes these functions available by default, so we should
not treat them as GNU-specific and restrict them to _GNU_SOURCE.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Prevent sh from interpreting a user string as shell options if it
starts with '-' or '+'. Since the version of /bin/sh used for testing
system() is different from the full-fledged system /bin/sh add support
to it for handling "--" after "-c". Add a testcase to ensure the
expected behavior.
Signed-off-by: Joe Simmons-Talbott <josimmon@redhat.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Instead of using a special ELF section along with a linker script
directive to put the IO vtables within the RELRO section, the libio
vtables are all moved to an array marked as data.relro (so linker
will place in the RELRO segment without the need of extra directives).
To avoid static linking namespace issues and including all vtable
referenced objects, all required function pointers are set to weak alias.
Checked on x86_64-linux-gnu, i686-linux-gnu, and aarch64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Instead define the required fields in system dependend files. The only
system dependent definition is FILENAME_MAX, which should match POSIX
PATH_MAX, and it is obtained from either kernel UAPI or mach headers.
Currently set pre-defined value from current kernels.
It avoids a circular dependendy when including stdio.h in
gen-as-const-headers files.
Checked on x86_64-linux-gnu and i686-linux-gnu
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
They are both used by __libc_freeres to free all library malloc
allocated resources to help tooling like mtrace or valgrind with
memory leak tracking.
The current scheme uses assembly markers and linker script entries
to consolidate the free routine function pointers in the RELRO segment
and to be freed buffers in BSS.
This patch changes it to use specific free functions for
libc_freeres_ptrs buffers and call the function pointer array directly
with call_function_static_weak.
It allows the removal of both the internal macros and the linker
script sections.
Checked on x86_64-linux-gnu, i686-linux-gnu, and aarch64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
C2x adds binary integer constants starting with 0b or 0B, and supports
those constants for the %i scanf format (in addition to the %b format,
which isn't yet implemented for scanf in glibc). Implement that scanf
support for glibc.
As with the strtol support, this is incompatible with previous C
standard versions, in that such an input string starting with 0b or 0B
was previously required to be parsed as 0 (with the rest of the input
potentially matching subsequent parts of the scanf format string).
Thus this patch adds 12 new __isoc23_* functions per long double
format (12, 24 or 36 depending on how many long double formats the
glibc configuration supports), with appropriate header redirection
support (generally very closely following that for the __isoc99_*
scanf functions - note that __GLIBC_USE (DEPRECATED_SCANF) takes
precedence over __GLIBC_USE (C2X_STRTOL), so the case of GNU
extensions to C89 continues to get old-style GNU %a and does not get
this new feature). The function names would remain as __isoc23_* even
if C2x ends up published in 2024 rather than 2023.
When scanf %b support is added, I think it will be appropriate for all
versions of scanf to follow C2x rules for inputs to the %b format
(given that there are no compatibility concerns for a new format).
Tested for x86_64 (full glibc testsuite). The first version was also
tested for powerpc (32-bit) and powerpc64le (stdio-common/ and wcsmbs/
tests), and with build-many-glibcs.py.
Almost all uses of rawmemchr find the end of a string. Since most targets use
a generic implementation, replacing it with strchr is better since that is
optimized by compilers into strlen (s) + s. Also fix the generic rawmemchr
implementation to use a cast to unsigned char in the if statement.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The __printf_buffer_flush_dprintf function needs to record that
the buffer has been written before reusing it. Without this
accounting, dprintf always returns zero.
Fixes commit 8ece45e4f5
("libio: Convert __vdprintf_internal to buffers").
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
This shows up as an assertion failure when sprintf is called with
a specifier like "%.8g" and libquadmath is linked in:
Fatal glibc error: printf_buffer_as_file.c:31
(__printf_buffer_as_file_commit): assertion failed:
file->stream._IO_write_ptr <= file->next->write_end
Fix this by detecting pointer wraparound in __vsprintf_internal
and saturate the addition to the end of the address space instead.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Always null-terminate the buffer and set E2BIG if the buffer is too
small. This fixes bug 27857.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The internal buffer size is set to 2048 bytes. This is less than
the original BUFSIZ value used by buffered_vfprintf before
the conversion, but it hopefully covers all cases where write
boundaries matter.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The buffer resizing algorithm is slightly different. The initial
buffer is on the stack, and small buffers are directly allocated
on the heap using the exact required size. The overhead of the
additional copy is compensated by the lowered setup cost for buffers
compared to libio streams.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
vfprintf is entangled with vfwprintf (of course), __printf_fp,
__printf_fphex, __vstrfmon_l_internal, and the strfrom family of
functions. The latter use the internal snprintf functionality,
so vsnprintf is converted as well.
The simples conversion is __printf_fphex, followed by
__vstrfmon_l_internal and __printf_fp, and finally
__vfprintf_internal and __vfwprintf_internal. __vsnprintf_internal
and strfrom* are mostly consuming the new interfaces, so they
are comparatively simple.
__printf_fp is a public symbol, so the FILE *-based interface
had to preserved.
The __printf_fp rewrite does not change the actual binary-to-decimal
conversion algorithm, and digits are still not emitted directly to
the target buffer. However, the staging buffer now uses bytes
instead of wide characters, and one buffer copy is eliminated.
The changes are at least performance-neutral in my testing.
Floating point printing and snprintf improved measurably, so that
this Lua script
for i=1,5000000 do
print(i, i * math.pi)
end
runs about 5% faster for me. To preserve fprintf performance for
a simple "%d" format, this commit has some logic changes under
LABEL (unsigned_number) to avoid additional function calls. There
are certainly some very easy performance improvements here: binary,
octal and hexadecimal formatting can easily avoid the temporary work
buffer (the number of digits can be computed ahead-of-time using one
of the __builtin_clz* built-ins). Decimal formatting can use a
specialized version of _itoa_word for base 10.
The existing (inconsistent) width handling between strfmon and printf
is preserved here. __print_fp_buffer_1 would have to use
__translated_number_width to achieve ISO conformance for printf.
Test expectations in libio/tst-vtables-common.c are adjusted because
the internal staging buffer merges all virtual function calls into
one.
In general, stack buffer usage is greatly reduced, particularly for
unbuffered input streams. __printf_fp can still use a large buffer
in binary128 mode for %g, though.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
clang emits an warning when a double alias redirection is used, to warn
the the original symbol will be used even when weak definition is
overridden. However, this is a common pattern for weak_alias, where
multiple alias are set to same symbol.
Reviewed-by: Fangrui Song <maskray@google.com>
In the future, this will result in a compilation failure if the
macros are unexpectedly undefined (due to header inclusion ordering
or header inclusion missing altogether).
Assembler sources are more difficult to convert. In many cases,
they are hand-optimized for the mangling and no-mangling variants,
which is why they are not converted.
sysdeps/s390/s390-32/__longjmp.c and sysdeps/s390/s390-64/__longjmp.c
are special: These are C sources, but most of the implementation is
in assembler, so the PTR_DEMANGLE macro has to be undefined in some
cases, to match the assembler style.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This allows us to define a generic no-op version of PTR_MANGLE and
PTR_DEMANGLE. In the future, we can use PTR_MANGLE and PTR_DEMANGLE
unconditionally in C sources, avoiding an unintended loss of hardening
due to missing include files or unlucky header inclusion ordering.
In i386 and x86_64, we can avoid a <tls.h> dependency in the C
code by using the computed constant from <tcb-offsets.h>. <sysdep.h>
no longer includes these definitions, so there is no cyclic dependency
anymore when computing the <tcb-offsets.h> constants.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The Z modifier is a nonstandard synonymn for z (that predates z
itself) and compiler might issue an warning for in invalid
conversion specifier.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Compilers may not be able to apply asm redirections to functions after
these functions are used for the first time, e.g. clang 13.
Fix [BZ #27087] by applying all long double-related asm redirections
before using functions in bits/stdio.h.
However, as these asm redirections depend on the declarations provided
by libio/bits/stdio2.h, this header was split in 2:
- libio/bits/stdio2-decl.h contains all function declarations;
- libio/bits/stdio2.h remains with the remaining contents, including
redirections.
This also adds the access attribute to __vsnprintf_chk that was missing.
Tested with build-many-glibcs.py.
Reviewed-by: Paul E. Murphy <murphyp@linux.ibm.com>
In general, _IO_str_overflow returns the character passed as an argument
on success. However, if flush-only operation is requested by passing
EOF, returning EOF looks like an error, and the caller cannot tell
whether the operation was successful or not.
_IO_wstr_overflow had the same bug regarding WEOF.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The _IO_wfile_overflow does not check if the write pointer for wide
data is valid before access, different than _IO_file_overflow. This
leads to crash on some cases, as described by bug 28828.
The minimal sequence to produce the crash was:
#include <stdio.h>
#include <wchar.h>
int main (int ac, char **av)
{
setvbuf (stdout, NULL, _IOLBF, 0);
fgetwc (stdin);
fputwc (10, stdout); /*CRASH HERE!*/
return 0;
}
The "fgetwc(stdin);" is necessary since it triggers the bug by setting
the flag _IO_CURRENTLY_PUTTING on stdout indirectly (file wfileops.c,
function _IO_wfile_underflow, line 213).
Signed-off-by: Jose Bollo <jobol@nonadev.net>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
In _FORTIFY_SOURCE=3, the size expression may be non-constant,
resulting in branches in the inline functions remaining intact and
causing a tiny overhead. Clang (and in future, gcc) make sure that
the -1 case is always safe, i.e. any comparison of the generated
expression with (size_t)-1 is always false so that bit is taken care
of. The rest is avoidable since we want the _chk variant whenever we
have a size expression and it's not -1.
Rework the conditionals in a uniform way to clearly indicate two
conditions at compile time:
- Either the size is unknown (-1) or we know at compile time that the
operation length is less than the object size. We can call the
original function in this case. It could be that either the length,
object size or both are non-constant, but the compiler, through
range analysis, is able to fold the *comparison* to a constant.
- The size and length are known and the compiler can see at compile
time that operation length > object size. This is valid grounds for
a warning at compile time, followed by emitting the _chk variant.
For everything else, emit the _chk variant.
This simplifies most of the fortified function implementations and at
the same time, ensures that only one call from _chk or the regular
function is emitted.
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
In the context of a function definition, the size hints imply that the
size of an object pointed to by one parameter is another parameter.
This doesn't make sense for the fortified versions of the functions
since that's the bit it's trying to validate.
This is harmless with __builtin_object_size since it has fairly simple
semantics when it comes to objects passed as function parameters.
With __builtin_dynamic_object_size we could (as my patchset for gcc[1]
already does) use the access attribute to determine the object size in
the general case but it misleads the fortified functions.
Basically the problem occurs when access attributes are present on
regular functions that have inline fortified definitions to generate
_chk variants; the attributes get inherited by these definitions,
causing problems when analyzing them. For example with poll(fds, nfds,
timeout), nfds is hinted using the __attr_access as being the size of
fds.
Now, when analyzing the inline function definition in bits/poll2.h, the
compiler sees that nfds is the size of fds and tries to use that
information in the function body. In _FORTIFY_SOURCE=3 case, where the
object size could be a non-constant expression, this information results
in the conclusion that nfds is the size of fds, which defeats the
purpose of the implementation because we're trying to check here if nfds
does indeed represent the size of fds. Hence for this case, it is best
to not have the access attribute.
With the attributes gone, the expression evaluation should get delayed
until the function is actually inlined into its destinations.
Disable the access attribute for fortified function inline functions
when building at _FORTIFY_SOURCE=3 to make this work better. The
access attributes remain for the _chk variants since they can be used
by the compiler to warn when the caller is passing invalid arguments.
[1] https://gcc.gnu.org/pipermail/gcc-patches/2021-October/581125.html
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
The test expects stdin to be a file which is not the case when running
tests over ssh where stdin is piped in.
The test fails with:
error: xlseek.c:27: lseek64 (0, 0, 1): Illegal seek
Update the test to create a temporary file and use that to perform the
test.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
C2X adds a macro _PRINTF_NAN_LEN_MAX to <stdio.h>, giving the maximum
length of printf output for a NaN. glibc never includes an
n-char-sequence in its printf output for NaNs, so the correct value
for glibc is 4 ("-nan" or "-NAN"); define the macro accordingly.
This patch makes the macro definition conditional on __GLIBC_USE
(ISOC2X), as is generally done with features from new standard
versions. The name is in the implementation namespace for older
standards, so it would also be possible to define it unconditionally.
Tested for x86_64.
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Remove all malloc hook uses from core malloc functions and move it
into a new library libc_malloc_debug.so. With this, the hooks now no
longer have any effect on the core library.
libc_malloc_debug.so is a malloc interposer that needs to be preloaded
to get hooks functionality back so that the debugging features that
depend on the hooks, i.e. malloc-check, mcheck and mtrace work again.
Without the preloaded DSO these debugging features will be nops.
These features will be ported away from hooks in subsequent patches.
Similarly, legacy applications that need hooks functionality need to
preload libc_malloc_debug.so.
The symbols exported by libc_malloc_debug.so are maintained at exactly
the same version as libc.so.
Finally, static binaries will no longer be able to use malloc
debugging features since they cannot preload the debugging DSO.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
For the legacy ABI with supports 32-bit time_t it calls the 64-bit
time directly, since the LFS symbols calls the 64-bit time_t ones
internally.
Checked on i686-linux-gnu and x86_64-linux-gnu.
Reviewed-by: Lukasz Majewski <lukma@denx.de>
Some symbols have explicit versioned_symbol or compat_symbol markers
in the sources, but no corresponding entry in the Versions files.
This presently works because the local: * directive is only applied
to the base version.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The symbol has never been exported, so no compatibility symbol is
needed. Removing this file prevents ld from creation an exported
symbol in case GLIBC_2_0 expands to a symbol version which
does not have a local: *; directive in the symbol version map file.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
To help detect common kinds of memory (and other resource) management
bugs, GCC 11 adds support for the detection of mismatched calls to
allocation and deallocation functions. At each call site to a known
deallocation function GCC checks the set of allocation functions
the former can be paired with and, if the two don't match, issues
a -Wmismatched-dealloc warning (something similar happens in C++
for mismatched calls to new and delete). GCC also uses the same
mechanism to detect attempts to deallocate objects not allocated
by any allocation function (or pointers past the first byte into
allocated objects) by -Wfree-nonheap-object.
This support is enabled for built-in functions like malloc and free.
To extend it beyond those, GCC extends attribute malloc to designate
a deallocation function to which pointers returned from the allocation
function may be passed to deallocate the allocated objects. Another,
optional argument designates the positional argument to which
the pointer must be passed.
This change is the first step in enabling this extended support for
Glibc.
This change continues the improvements to compile-time out of bounds
checking by decorating more APIs with either attribute access, or by
explicitly providing the array bound in APIs such as tmpnam() that
expect arrays of some minimum size as arguments. (The latter feature
is new in GCC 11.)
The only effects of the attribute and/or the array bound is to check
and diagnose calls to the functions that fail to provide a sufficient
number of elements, and the definitions of the functions that access
elements outside the specified bounds. (There is no interplay with
_FORTIFY_SOURCE here yet.)
Tested with GCC 7 through 11 on x86_64-linux.
No new symbol version is required because there was a forwarder.
The symbol has been moved using scripts/move-symbol-to-libc.py.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
So that text_set_element/data_set_element/bss_set_element defined
variables will be retained by the linker.
Note: 'used' and 'retain' are orthogonal: 'used' makes sure the variable
will not be optimized out; 'retain' prevents section garbage collection
if the linker support SHF_GNU_RETAIN.
GNU ld 2.37 and LLD 13 will support -z start-stop-gc which allow C
identifier name sections to be GCed even if there are live
__start_/__stop_ references.
Without the change, there are some static linking problems, e.g.
_IO_cleanup (libio/genops.c) may be discarded by ld --gc-sections, so
stdout is not flushed on exit.
Note: GCC may warning 'retain' attribute ignored while __has_attribute(retain)
is 1 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99587).
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
About a decade ago, I accidentally wrote the GPLv3 license text on the
test case when the rest of glibc source is LGPL v2.1 or later. As
original author of the test (and there are no other legally
significant changes to the test) I propose to update the license text
to be consistent with the project.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
An experimental build of GCC 11 with an enhanced -Warray-bounds
reports a bug in IO_wdefault_doallocate where the function forms
an invalid past-the-end pointer to an allocated wchar_t buffer
by failingf to consider the scaling by sizeof (wchar_t).
The fix path below corrects this problem. It keeps the buffer
size the same as opposed to increasing it according to what other
code like it does.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
It replaces the internal usage of __{f,l}xstat{at}{64} with the
__{f,l}stat{at}{64}. It should not change the generate code since
sys/stat.h explicit defines redirections to internal calls back to
xstat* symbols.
Checked with a build for all affected ABIs. I also check on
x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: Lukasz Majewski <lukma@denx.de>
If build with optimization, stdio.h and sys/stat.h are defining some inlining
functions. This leads to test fails if glibc is build with the following
commands. (Note that the conformtests usually builds without optimization or
other CFLAGS):
<glibc>/configure CC="gcc -O3" --prefix=/usr
make
make subdirs=conform check
- FAIL: conform/XPG4/stdio.h/conform
- FAIL: conform/XPG42/stdio.h/conform
out-files:
...
PASSCOMBINED: Availability of variable optopt
PASSCOMBINED: Type of variable optopt
Namespace violation: "getc_unlocked"
Namespace violation: "getchar_unlocked"
Namespace violation: "putc_unlocked"
Namespace violation: "putchar_unlocked"
FAIL: Namespace of <stdio.h>
----------------------------------------------------------------------------
Total number of tests : 168
Number of failed tests : 1
Number of xfailed tests : 0
Number of skipped tests : 0
- FAIL: conform/POSIX2008/sys/stat.h/conform
out-file:
...
PASSCOMBINED: Availability of function utimensat
PASSCOMBINED: Type of function utimensat
Namespace violation: "mknodat"
FAIL: Namespace of <sys/stat.h>
----------------------------------------------------------------------------
Total number of tests : 97
Number of failed tests : 1
Number of xfailed tests : 0
Number of skipped tests : 0
For getc_unlocked, getchar_unlocked, putc_unlocked, putchar_unlocked in stdio.h,
those are defined "# ifdef __USE_POSIX" instead of "#ifdef __USE_POSIX199506"
for the non-inlining declaration. See also
"Bug 20014 - stdio.h namespace for pre-threads POSIX"
(https://sourceware.org/bugzilla/show_bug.cgi?id=20014).
For mknodat in sys/stat.h, those are defined "# ifdef __USE_ATFILE" instead of
the additional guard "# if defined __USE_MISC || defined __USE_XOPEN_EXTENDED".
__nss_readline supersedes it. This reverts part of commit
3f5e3f5d06 ("libio: Implement
internal function __libc_readline_unlocked"). The internal
aliases __fseeko64 and __ftello64 are preserved because
they are needed by __nss_readline as well.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The symbol is deprecated by strerror since its usage imposes some issues
such as copy relocations.
Its internal name is also changed to _sys_errlist_internal to avoid
static linking usage. The compat code is also refactored by removing
the over enginered errlist-compat.c generation from manual entried and
extra comment token in linker script file. It disantangle the code
generation from manual and simplify both Linux and Hurd compat code.
The definitions from errlist.c are moved to errlist.h and a new test
is added to avoid a new errno entry without an associated one in manual.
Checked on x86_64-linux-gnu and i686-linux-gnu. I also run a check-abi
on all affected platforms.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Adds the access attribute newly introduced in GCC 10 to the subset of
function declarations that are already covered by _FORTIFY_SOURCE and
that don't have corresponding GCC built-in equivalents.
Reviewed-by: DJ Delorie <dj@redhat.com>
Improve the commentary to aid future developers who will stumble
upon this novel, yet not always perfect, mechanism to support
alternative formats for long double.
Likewise, rename __LONG_DOUBLE_USES_FLOAT128 to
__LDOUBLE_REDIRECTS_TO_FLOAT128_ABI now that development work
has settled down. The command used was
git grep -l __LONG_DOUBLE_USES_FLOAT128 ':!./ChangeLog*' | \
xargs sed -i 's/__LONG_DOUBLE_USES_FLOAT128/__LDOUBLE_REDIRECTS_TO_FLOAT128_ABI/g'
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
The new type struct fd_to_filename makes the allocation of the
backing storage explicit.
Hurd uses /dev/fd, not /proc/self/fd.
Co-Authored-By: Paul Eggert <eggert@cs.ucla.edu>
All functions that have a format string, which can consume a long double
argument, must have one version for each long double format supported on
a platform. On powerpc64le, these functions currently have two versions
(i.e.: long double with the same format as double, and long double with
IBM Extended Precision format). Support for a third long double format
option (i.e. long double with IEEE long double format) is being prepared
and all the aforementioned functions now have a third version (not yet
exported on the master branch, but the code is in).
For these functions to get selected (during build time), references to
them in user programs (or dependent libraries) must get redirected to
the aforementioned new versions of the functions. This patch installs
the header magic required to perform such redirections.
Notice, however, that since the redirections only happen when
__LONG_DOUBLE_USES_FLOAT128 is set to 1, and no platform (including
powerpc64le) currently does it, no redirections actually happen.
Redirections and the exporting of the new functions will happen at the
same time (when powerpc64le adds ldbl-128ibm-compat to their Implies.
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
Reviewed-by: Paul E. Murphy <murphyp@linux.vnet.ibm.com>
It enables and disables cancellation with pthread_setcancelstate
before calling the waitpid. It simplifies the waitpid implementation
for architectures that do not provide either __NR_waitpid or
__NR_wait4.
Checked on x86_64-linux-gnu.
Commit c402355dfa ("libio: Disable
vtable validation in case of interposition [BZ #23313]") only covered
the interposable glibc 2.1 handles, in libio/stdfiles.c. The
parallel code in libio/oldstdfiles.c needs similar detection logic.
Fixes (again) commit db3476aff1
("libio: Implement vtable verification [BZ #20191]").
Change-Id: Ief6f9f17e91d1f7263421c56a7dc018f4f595c21
When the commit
commit 03992356e6
Author: Zack Weinberg <zackw@panix.com>
Date: Sat Feb 10 11:58:35 2018 -0500
Use C99-compliant scanf under _GNU_SOURCE with modern compilers.
added the DEPRECATED_SCANF macro to select when redirections of *scanf
functions to their ISO C99 compliant versions should happen, it
accidentally missed doing it for vfwscanf, vwscanf, and vswscanf.
Tested for powerpc64le and with build-many-glibcs (i686-linux-gnu and
nios2-linux-gnu are failing with current master, and with this patch,
but I didn't see a regression).
Change-Id: I706b344a3fb50be017cdab9251d9da18a3ba8c60
libio can only deal with gconv conversions which consist of a single
step. Not using __gconv_info simplifies the data structures somewhat.
This eliminates a new GCC 10 warning about subscribing an inner
zero-length array.
Tested on x86_64-linux-gnu with mainline GCC. Built with
build-many-glibcs.py, also with mainline GCC. Due to GCC PR 92039,
there are failures left on 32-bit architectures with float128 support.
Change-Id: I8b4c489b619a53154712ff32e1b6f13bb92d4203
The changes introduce a memory leak for gconv steps arrays whose
first element is an internal conversion, which has a fixed
reference count which is not decremented. As a result, after the
change in commit 50ce3eae5b, the steps
array is never freed, resulting in an unbounded memory leak.
This reverts commit 50ce3eae5b
("gconv: Check reference count in __gconv_release_cache
[BZ #24677]") and commit 7e740ab2e7
("libio: Fix gconv-related memory leak [BZ #24583]"). It
reintroduces bug 24583. (Bug 24677 was just a regression caused by
the second commit.)
On powerpc64le, long double can currently take two formats: the same as
double (-mlong-double-64) or IBM Extended Precision (default with
-mlong-double-128 or explicitly with -mabi=ibmlongdouble). The internal
implementation of printf-like functions is aware of these possibilities
and properly parses floating-point values from the variable arguments,
before making calls to __printf_fp and __printf_fphex. These functions
are also aware of the format possibilities and know how to convert both
formats to string.
When library support for TS 18661-3 was added to glibc, __printf_fp and
__printf_fphex were extended with support for an additional type
(__float128/_Float128) with a different format (binary128). Now that
powerpc64le is getting support for its third long double format, and
taking into account that this format is the same as the format of
__float128/_Float128, this patch extends __vfprintf_internal to properly
call __printf_fp and __printf_fphex with this new format.
Tested for powerpc64le (with additional patches to actually enable the
use of these preparations) and for x86_64.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Commit a601b74d31 aka glibc-2.23~693
("In preparation for fixing BZ#16734, fix failure in misc/tst-error1-mem
when _G_HAVE_MMAP is turned off.") introduced a regression:
_IO_unbuffer_all now invokes _IO_wsetb to free wide buffers of all
files, including legacy standard files which are small statically
allocated objects that do not have wide buffers and the _mode member,
causing memory corruption.
Another memory corruption in _IO_unbuffer_all happens when -1
is assigned to the _mode member of legacy standard files that
do not have it.
[BZ #24228]
* libio/genops.c (_IO_unbuffer_all)
[SHLIB_COMPAT (libc, GLIBC_2_0, GLIBC_2_1)]: Do not attempt to free wide
buffers and access _IO_FILE_complete members of legacy libio streams.
* libio/tst-bz24228.c: New file.
* libio/tst-bz24228.map: Likewise.
* libio/Makefile [build-shared] (tests): Add tst-bz24228.
[build-shared] (generated): Add tst-bz24228.mtrace and
tst-bz24228.check.
[run-built-tests && build-shared] (tests-special): Add
$(objpfx)tst-bz24228-mem.out.
(LDFLAGS-tst-bz24228, tst-bz24228-ENV): New variables.
($(objpfx)tst-bz24228-mem.out): New rule.
struct gconv_fcts for the C locale is statically allocated,
and __gconv_close_transform deallocates the steps object.
Therefore this commit introduces __wcsmbs_close_conv to avoid
freeing the statically allocated steps objects.
The codecvt vtable is not a real vtable because it also contains the
conversion state data. Furthermore, wide stream support was added to
GCC 3.0, after a C++ ABI bump, so there is no compatibility
requirement with libstdc++.
This change removes several unmangled function pointers which could
be used with a corrupted FILE object to redirect execution. (libio
vtable verification did not cover the codecvt vtable.)
Reviewed-by: Yann Droneaud <ydroneaud@opteya.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
When computing the length of the converted part of the stdio buffer, use
the number of consumed wide characters, not the (negative) distance to the
end of the wide buffer.
These variables are only used to determine if a stdio stream is
a pre-allocated stream, but it is possible to do so by comparing
a FILE * to all pre-allocated stream objects. As a result, it is
not necessary to keep those pointers in separate variables.
Behavior with symbol interposition is unchanged because _IO_stdin_,
_IO_stdout_, _IO_stderr_ are exported, and refer to objects outside of
libc if symbol interposition or copy relocations are involved. (The
removed variables _IO_stdin, _IO_stdout, _IO_stderr were not exported,
of course.)
One group of warnings seen building glibc with -Wextra is -Wempty-body
warnings about an 'if' body (or in one case an 'else' body) that is
just a semicolon, "warning: suggest braces around empty body in an
'if' statement [-Wempty-body]" - I think the point of the warning
being to make it more visible whether an 'if' body is actually present
or not.
This patch fixes such warnings in glibc. There's one place, with a
semicolon at the end of a comment, where this is clearly making the
presence of an 'else' body more visible. The other cases involve
macro definitions expanding to nothing. While there's no issue there
with visibility at the call sites, I think it's still cleaner to have
a macro that expands to something nonempty appropriate for the context
- so do {} while (0) if it's only intended to be usable as a
statement, or ((void) 0) where the macro definition is an alternative
to a call to a function returning void, so this patch makes those
changes.
Tested for x86_64.
* catgets/gencat.c (normalize_line): Use braces around empty
'else' body.
* include/stap-probe.h [!USE_STAP_PROBE && !__ASSEMBLER__]
(STAP_PROBE0): Use do {} while (0) for do-nothing definition.
[!USE_STAP_PROBE && !__ASSEMBLER__] (STAP_PROBE1): Likewise.
[!USE_STAP_PROBE && !__ASSEMBLER__] (STAP_PROBE2): Likewise.
[!USE_STAP_PROBE && !__ASSEMBLER__] (STAP_PROBE3): Likewise.
[!USE_STAP_PROBE && !__ASSEMBLER__] (STAP_PROBE4): Likewise.
* libio/libio.h (_IO_funlockfile): Use ((void) 0) for do-nothing
definition.
On big endian systems the test fails with:
tst-bz24153.c:88: numeric comparison failure
left: 1660944385 (0x63000001); from: ch
right: 99 (0x63); from: L'c'
tst-bz24153.c:90: numeric comparison failure
left: 1677721601 (0x64000001); from: ch
right: 100 (0x64); from: L'd'
error: 2 test failures
One 'char' ("%c") is stored to the 'wchar_t *':
ch = 0x00000001 | 0x63000000
This patch is using "%lc" as format specifier to read a wchar_t.
ChangeLog:
* libio/tst-bz24153.c (wide): Use wide char format specifier.
GLIBC explicitly allows one to assign a new FILE pointer to stdout and
other standard streams. printf and wprintf were honouring assignment to
stdout and using the new value, but puts, putchar, and wide char variants
did not.
The stdout part is fixed here. The stdin part will be fixed in a followup.
The only difference between noncompliant and C99-compliant scanf is
that the former accepts the archaic GNU extension '%as' (also %aS and
%a[...]) meaning to allocate space for the input string with malloc.
This extension conflicts with C99's use of %a as a format _type_
meaning to read a floating-point number; POSIX.1-2008 standardized
equivalent functionality using the modifier letter 'm' instead (%ms,
%mS, %m[...]).
The extension was already disabled in most conformance modes:
specifically, any mode that doesn't involve _GNU_SOURCE and _does_
involve either strict conformance to C99 or loose conformance to both
C99 and POSIX.1-2001 would get the C99-compliant scanf. With
compilers new enough to use -std=gnu11 instead of -std=gnu89, or
equivalent, that includes the default mode.
With this patch, we now provide C99-compliant scanf in all
configurations except when _GNU_SOURCE is defined *and*
__STDC_VERSION__ or __cplusplus (whichever is relevant) indicates
C89/C++98. This leaves the old scanf available under e.g. -std=c89
-D_GNU_SOURCE, but removes it from e.g. -std=gnu11 -D_GNU_SOURCE (it
was already not present under -std=gnu11 without -D_GNU_SOURCE) and
from -std=gnu89 without -D_GNU_SOURCE.
There needs to be an internal override so we can compile the
noncompliant scanf itself. This is the same problem we had when we
removed 'gets' from _GNU_SOURCE and it's dealt with the same way:
there's a new __GLIBC_USE symbol, DEPRECATED_SCANF, which defaults to
off under the appropriate conditions for external code, but can be
overridden by individual files within stdio.
We also run into problems with PLT bypass for internal uses of sscanf,
because libc_hidden_proto uses __REDIRECT and so does the logic in
stdio.h for choosing which implementation of scanf to use; __REDIRECT
isn't transitive, so include/stdio.h needs to bridge the gap with a
macro. As far as I can tell, sscanf is the only function in this
family that's internally called by unrelated code.
Finally, there are several tests in stdio-common that use the
extension. bug21.c is a regression test for a crash; it still
exercises the relevant code when changed to use %ms instead of %as.
scanf14.c through scanf17.c are more complicated since they are
actually testing the subtleties of the extension - under what
circumstances is 'a' treated as a modifier letter, etc. I changed all
of them to use %ms instead of %as as well, but duplicated scanf14.c
and scanf16.c as scanf14a.c and scanf16a.c. These still use %as and
are compiled with -std=gnu89 to access the old extension. A bunch of
diagnostic overrides and manual workarounds for the old stdio.h
behavior become unnecessary. Yay!
* include/features.h (__GLIBC_USE_DEPRECATED_SCANF): New __GLIBC_USE
parameter. Only use deprecated scanf when __USE_GNU is defined
and __STDC_VERSION__ is less than 199901L or __cplusplus is less
than 201103L, whichever is relevant for the language being compiled.
* libio/stdio.h, libio/bits/stdio-ldbl.h: Decide whether to redirect
scanf, fscanf, sscanf, vscanf, vfscanf, and vsscanf to their
__isoc99_ variants based only on __GLIBC_USE (DEPRECATED_SCANF).
* wcsmbs/wchar.h: wcsmbs/bits/wchar-ldbl.h: Likewise for
wscanf, fwscanf, swscanf, vwscanf, vfwscanf, and vswscanf.
* libio/iovsscanf.c
* libio/fwscanf.c
* libio/iovswscanf.c
* libio/swscanf.c
* libio/vscanf.c
* libio/vwscanf.c
* libio/wscanf.c
* stdio-common/fscanf.c
* stdio-common/scanf.c
* stdio-common/vfscanf.c
* stdio-common/vfwscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-compat.c
* sysdeps/ieee754/ldbl-opt/nldbl-fscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-fwscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-iovfscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-scanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-sscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-swscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-vfscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-vfwscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-vscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-vsscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-vswscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-vwscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-wscanf.c:
Override __GLIBC_USE_DEPRECATED_SCANF to 1.
* stdio-common/sscanf.c: Likewise. Remove ldbl_hidden_def for __sscanf.
* stdio-common/isoc99_sscanf.c: Add libc_hidden_def for __isoc99_sscanf.
* include/stdio.h: Provide libc_hidden_proto for __isoc99_sscanf,
not sscanf.
[!__GLIBC_USE (DEPRECATED_SCANF)]: Define sscanf as __isoc99_scanf
with a preprocessor macro.
* stdio-common/bug21.c, stdio-common/scanf14.c:
Use %ms instead of %as, %mS instead of %aS, %m[] instead of %a[];
remove DIAG_IGNORE_NEEDS_COMMENT for -Wformat.
* stdio-common/scanf16.c: Likewise. Add __attribute__ ((format (scanf)))
to xscanf, xfscanf, xsscanf.
* stdio-common/scanf14a.c: New copy of scanf14.c which still uses
%as, %aS, %a[]. Remove DIAG_IGNORE_NEEDS_COMMENT for -Wformat.
* stdio-common/scanf16a.c: New copy of scanf16.c which still uses
%as, %aS, %a[]. Add __attribute__ ((format (scanf))) to xscanf,
xfscanf, xsscanf.
* stdio-common/scanf15.c, stdio-common/scanf17.c: No need to
override feature selection macros or provide definitions of u_char etc.
* stdio-common/Makefile (tests): Add scanf14a and scanf16a.
(CFLAGS-scanf15.c, CFLAGS-scanf17.c): Remove.
(CFLAGS-scanf14a.c, CFLAGS-scanf16a.c): New. Compile these files
with -std=gnu89.
According to ISO C99, passing the same buffer as source and destination
to sprintf, snprintf, vsprintf, or vsnprintf has undefined behavior.
Until the commit
commit 4e2f43f842
Author: Zack Weinberg <zackw@panix.com>
Date: Wed Mar 7 14:32:03 2018 -0500
Use PRINTF_FORTIFY instead of _IO_FLAGS2_FORTIFY (bug 11319)
a call to sprintf or vsprintf with overlapping buffers, for instance
vsprintf (buf, "%sTEXT", buf), would append `TEXT' into buf, while a
call to snprintf or vsnprintf would override the contents of buf.
After the aforementioned commit, the behavior of sprintf and vsprintf
changed (so that they also override the contents of buf).
This patch reverts this behavioral change, because it will likely break
applications that rely on the previous behavior, even though it is
undefined by ISO C. As noted by Szabolcs Nagy, this is used in SPEC2017
507.cactuBSSN_r/src/PUGH/PughUtils.c:
sprintf(mess," Size:");
for (i=0;i<dim+1;i++)
{
sprintf(mess,"%s %d",mess,pughGH->GFExtras[dim]->nsize[i]);
}
More important to notice is the fact that the overwriting of the
destination buffer is not the only behavior affected by the refactoring.
Before the refactoring, sprintf and vsprintf would use _IO_str_jumps,
whereas __sprintf_chk and __vsprintf_chk would use _IO_str_chk_jumps.
After the refactoring, all use _IO_str_chk_jumps, which would make
sprintf and vsprintf report buffer overflows and terminate the program.
This patch also reverts this behavior, by installing the appropriate
jump table for each *sprintf functions.
Apart from reverting the changes, this patch adds a test case that has
the old behavior hardcoded, so that regressions are noticed if something
else unintentionally changes the behavior.
Tested for powerpc64le.
Joseph Myers
Florian Weimer
Paul Eggert
Xi Ruoyao
Adam Jackson
Samuel Thibault
Joe Simmons-Talbott
Frédéric Bérat
Andreas Schwab
Paul Pluzhnikov
Sergey Bugaev
Adhemerval Zanella
Joseph Myers
Wilco Dijkstra
Tulio Magno Quites Machado Filho
José Bollo
Siddhesh Poyarekar
Stafford Horne
Martin Sebor
Fangrui Song
Martin Sebor
Stefan Liebler
Paul E. Murphy
Gabriel F. T. Gomes
Arjun Shankar
Dmitry V. Levin
Zack Weinberg
Gabriel F. T. Gomes