As discussed in bug 32369 and required by POSIX, the POSIX feature
fflush (NULL) should flush input files, not just output files. The
POSIX requirement is that "fflush() shall perform this flushing action
on all streams for which the behavior is defined above", and the
definition for input files is for "a stream open for reading with an
underlying file description, if the file is not already at EOF, and
the file is one capable of seeking".
Implement this requirement in glibc. (The underlying flushing
implementation is what deals with avoiding errors for seeking on an
unseekable file.)
Tested for x86_64.
As discussed in bug 12724 and required by POSIX, before an input file
(based on an underlying seekable file descriptor) is closed, fclose is
sometimes required to seek that file descriptor to the correct offset,
so that any other file descriptors sharing the underlying open file
description are left at that offset (as a motivating example, a script
could call a sequence of commands each of which processes some data
from (seekable) stdin using stdio; fclose needs to do this so that
each successive command can read exactly the data not handled by
previous commands), but glibc fails to do this.
The precise POSIX wording has changed a few times; in the 2024 edition
it's "If the file is not already at EOF, and the file is one capable
of seeking, the file offset of the underlying open file description
shall be set to the file position of the stream if the stream is the
active handle to the underlying file description.".
Add appropriate logic to _IO_new_file_close_it to handle this case. I
haven't made any attempt to test or change things in this area for the
"old" functions.
Note that there was a previous attempt to fix bug 12724, reverted in
commit eb6cbd249f. The fix version here
addresses the original test in that bug report without breaking the
one given in a subsequent comment in that bug report (which works with
glibc before the patch, but maybe was broken by the original fix that
was reverted).
The logic here tries to take care not to seek the file, even to its
newly computed current offset, if at EOF / possibly not the active
handle; even seeking to the current offset would be problematic
because of a potential race (fclose computes the current offset,
another thread or process with the active handle does its own seek,
fclose does a seek (not permitted by POSIX in this case) that loses
the effect of the seek on the active handle in another thread or
process). There are tests included for various cases of being or not
being the active handle, though there aren't tests for the potential
race condition.
Tested for x86_64.
As discussed in bug 5994 (plus duplicates), POSIX requires fflush
after ungetc to discard pushed-back characters but preserve the file
position indicator. For this purpose, each ungetc decrements the file
position indicator by 1; it is unspecified after ungetc at the start
of the file, and after ungetwc, so no special handling is needed for
either of those cases.
This is fixed with appropriate logic in _IO_new_file_sync. I haven't
made any attempt to test or change things in this area for the "old"
functions; the case of files using mmap is addressed in a subsequent
patch (and there seem to be no problems in this area with files opened
with fmemopen).
Tested for x86_64.
Test if the file-position is correctly updated when fwrite tries to
flush its internal cache but is not able to completely write all items.
Reviewed-by: DJ Delorie <dj@redhat.com>
When an error happens, fwrite is expected to return a value that is less
than nmemb. If this error happens while flushing its internal buffer,
fwrite is in a complex scenario: all the data might have been written to
the buffer, indicating a successful copy, but the buffer is expected to
be flushed and it was not.
POSIX.1-2024 states the following about errors on fwrite:
If an error occurs, the resulting value of the file-position indicator
for the stream is unspecified.
The fwrite() function shall return the number of elements successfully
written, which may be less than nitems if a write error is encountered.
With that in mind, this commit modifies _IO_new_file_write in order to
return the total number of bytes written via the file pointer. It also
modifies fwrite in order to use the new information and return the
correct number of bytes written even when sputn returns EOF.
Add 2 tests:
1. tst-fwrite-bz29459: This test is based on the reproducer attached to
bug 29459. In order to work, it requires to pipe stdout to another
process making it hard to reuse test-driver.c. This code is more
specific to the issue reported.
2. tst-fwrite-pipe: Recreates the issue by creating a pipe that is shared
with a child process. Reuses test-driver.c. Evaluates a more generic
scenario.
Co-authored-by: Florian Weimer <fweimer@redhat.com>
Reviewed-by: DJ Delorie <dj@redhat.com>
Adding some basic tests for fopen, testing different modes, stream
positioning and concurrent read/write operation on files.
Reviewed-by: DJ Delorie <dj@redhat.com>
When gawk was not built with MPFR, there's no mtrace output and those
tests FAIL. But we should make them UNSUPPORTED like other
tst-printf-format-* tests in the case.
Signed-off-by: Xi Ruoyao <xry111@xry111.site>
Reviewed-by: Sam James <sam@gentoo.org>
Reviewed-by: Andreas K Hüttel <dilfridge@gentoo.org>
clang-19 shows:
scanf13.c:28:40: error: 'sscanf' may overflow; destination buffer in argument 4 has size 8, but the corresponding specifier may require size 11 [-Werror,-Wfortify-source]
28 | "A%ms%10ms%4m[bcd]%4mcB", &sp1, &sp2, &sp3, &sp4) != 4)
| ^
scanf13.c:94:34: error: 'sscanf' may overflow; destination buffer in argument 3 has size 8, but the corresponding specifier may require size 2049 [-Werror,-Wfortify-source]
94 | if (sscanf (buf, "%2048ms%mc", &sp3, &sp4) != 2)
| ^
scanf13.c:110:61: error: 'sscanf' may overflow; destination buffer in argument 4 has size 8, but the corresponding specifier may require size 1501 [-Werror,-Wfortify-source]
110 | if (sscanf (buf, "%4mc%1500m[dr/]%548m[abc/d]%3mc", &sp1, &sp2, &sp3, &sp4)
| ^
scanf13.c:110:67: error: 'sscanf' may overflow; destination buffer in argument 5 has size 8, but the corresponding specifier may require size 549 [-Werror,-Wfortify-source]
110 | if (sscanf (buf, "%4mc%1500m[dr/]%548m[abc/d]%3mc", &sp1, &sp2, &sp3, &sp4)
clang does have some support to handle 'm' prefix for -Wformat; but it
lacks support for -Wfortify to understand that it is up to libc to
allocate the memory, and uses the pointer size instead to calculate
validity.
Since Clang doesn't support
DIAG_IGNORE_NEEDS_COMMENT (11, "-Wformat=");
and for unknown reasons, it doesn't warn the %#m specifier, suppress
-Wformat only for gcc in tst-sprintf-errno.c.
Co-Authored-By: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Sam James <sam@gentoo.org>
Clang issues the following warning:
tst-vfprintf-width-i18n.c:51:34: error: invalid conversion specifier '1'
[-Werror,-Wformat-invalid-specifier]
TEST_COMPARE (sprintf (buf, "%I16d", 12345), 16);
~~^
since it does not how to handle %I.
Reviewed-by: Sam James <sam@gentoo.org>
clang does not handle %Z on print, and just suppressing
-Wformat-invalid-specifier might trigger another warning for extra
arguments (since %Z is ignored). So suppress -Wformat-extra-args
as well.
For tst-fphex.c a heavy hammer is used since the printf is more
complex and clang throws a more generic warning.
Reviewed-by: Sam James <sam@gentoo.org>
Use
__attribute__ ((optnone))
instead of
__attribute__ ((optimize ("-O0")))
to disable optimization for Clang.
Co-Authored-By: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Sam James <sam@gentoo.org>
Clang 19 takes a very long time, it ran more than 27 minutes on Intel Core
i7-1195G7 before the process was killed, to compile bug28.c:
https://github.com/llvm/llvm-project/issues/120462
Exclude it when Clang is used for testing.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Sam James <sam@gentoo.org>
The C standard requires that ungetc guarantees at least one pushback,
but the malloc call to allocate the pushback buffer could fail, thus
violating that requirement. Fix this by adding a single byte pushback
buffer in the FILE struct that the pushback can fall back to if malloc
fails.
The side-effect is that if the initial malloc fails and the 1-byte
fallback buffer is used, future resizing (if it succeeds) will be
2-bytes, 4-bytes and so on, which is suboptimal but it's after a malloc
failure, so maybe even desirable.
A future optimization here could be to have the pushback code use the
single byte buffer first and only fall back to malloc for subsequent
calls.
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: Maciej W. Rozycki <macro@redhat.com>
GCC 4.9 issues an error for copysign in initializer:
In file included from tst-printf-format-p-double.c:20:0:
tst-printf-format-skeleton-double.c:29:3: error: initializer element is not a constant expression [-Werror]
{ -HUGE_VAL, -DBL_MAX, -DBL_MIN, copysign (0, -1), -NAN, NAN, 0, DBL_MIN,
^
since it can't fold "copysign (0, -1)". Replace copysign (0,-1) with -0.0.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Nameless function parameters have only been added to ISO C with the C23
revision of the language standard. Give names to the unused parameters
of the stub 'dladdr' implementation then so as to make compilation happy
with the earlier language definitions, fixing errors such as:
tst-printf-format-skeleton.c:374:9: error: parameter name omitted
374 | dladdr (const void *, Dl_info *)
reported by older compilers.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Wire vasprintf into test infrastructure for formatted printf output
specifiers.
Owing to mtrace logging these tests take amounts of time to complete
similar to those of corresponding asprintf tests, so set timeouts for
the tests accordingly, with a global default for all the vasprintf
tests, and then individual higher settings for double and long double
tests each.
Reviewed-by: DJ Delorie <dj@redhat.com>
Wire asprintf into test infrastructure for formatted printf output
specifiers.
Owing to mtrace logging of lots of memory allocation calls these tests
take a considerable amount of time to complete, except for the character
conversion, taking from 00m20s for 'tst-printf-format-as-s --direct s',
through 01m10s and 03m53s for 'tst-printf-format-as-char --direct i' and
'tst-printf-format-as-double --direct f' respectively, to 19m24s for
'tst-printf-format-as-ldouble --direct f', all in standalone execution
from NFS on a RISC-V FU740@1.2GHz system and with output redirected over
100Mbps network via SSH. It is with the skeleton's stub implementation
of dladdr(3); execution times with regular dladdr(3) are up to over
twice longer.
Set timeouts for the tests accordingly then, with a global default for
all the asprintf tests, and then individual higher settings for double
and long double tests each.
Reviewed-by: DJ Delorie <dj@redhat.com>
This is a collection of tests for formatted printf output specifiers
covering the d, i, o, u, x, and X integer conversions, the e, E, f, F,
g, and G floating-point conversions, the c character conversion, and the
s string conversion. Also the hh, h, l, and ll length modifiers are
covered with the integer conversions as is the L length modifier with
the floating-point conversions.
The -, +, space, #, and 0 flags are iterated over, as permitted by the
conversion handled, in tuples of 1..5, including tuples with repetitions
of 2, and combined with field width and/or precision, again as permitted
by the conversion. The resulting format string is then used to produce
output from respective sets of input data corresponding to the specific
conversion under test. POSIX extensions beyond ISO C are not used.
Output is produced in the form of records which include both the format
string (and width and/or precision where given in the form of separate
arguments) and the conversion result, and is verified with GNU AWK using
the format obtained from each such record against the reference value
also supplied, relying on the fact that GNU AWK has its own independent
implementation of format processing, striving to be ISO C compatible.
In the course of implementation I have determined that in the non-bignum
mode GNU AWK uses system sprintf(3) for the floating-point conversions,
defeating the objective of doing the verification against an independent
implementation. Additionally the bignum mode (using MPFR) is required
to correctly output wider integer and floating-point data. Therefore
for the conversions affected the relevant shell scripts sanity-check AWK
and terminate with unsupported status if the bignum mode is unavailable
for floating-point data or where data is output incorrectly.
The f and F floating-point conversions are build-time options for GNU
AWK, depending on the environment, so they are probed for before being
used. Similarly the a and A floating-point conversions, however they
are currently not used, see below. Also GNU AWK does not handle the b
or B integer conversions at all at the moment, as at 5.3.0. Support for
the a, A, b, and B conversions can however be easily added following the
approach taken for the f and F conversions.
Output produced by gawk for the a and A floating-point conversions does
not match one produced by us: insufficient precision is used where one
hasn't been explicitly given, e.g. for the negated maximum finite IEEE
754 64-bit value of -1.79769313486231570814527423731704357e+308 and "%a"
format we produce -0x1.fffffffffffffp+1023 vs gawk's -0x1.000000p+1024
and a different exponent is chosen otherwise, such as with "%.a" where
we output -0x2p+1023 vs gawk's -0x1p+1024 for the same value, or "%.20a"
where -0x1.fffffffffffff0000000p+1023 is our output, but gawk produces
-0xf.ffffffffffff80000000p+1020 instead. Consequently I chose not to
include a and A conversions in testing at this time.
And last but not least there are numerous corner cases that GNU AWK does
not handle correctly, which are worked around by explicit handling in
the AWK script. These are in particular:
- extraneous leading 0 produced for the alternative form with the o
conversion, e.g. { printf "%#.2o", 1 } produces "001" rather than
"01",
- unexpected 0 produced where no characters are expected for the input
of 0 and the alternative form with the precision of 0 and the integer
hexadecimal conversions, e.g. { printf "%#.x", 0 } produces "0" rather
than "",
- missing + character in the non-bignum mode only for the input of 0
with the + flag, precision of 0 and the signed integer conversions,
e.g. { printf "%+.i", 0 } produces "" rather than "+",
- missing space character in the non-bignum mode only for the input of 0
with the space flag, precision of 0 and the signed integer
conversions, e.g. { printf "% .i", 0 } produces "" rather than " ",
- for released gawk versions of up to 4.2.1 missing - character for the
input of -NaN with the floating-point conversions, e.g. { printf "%e",
"-nan" }' produces "nan" rather than "-nan",
- for released gawk versions from 5.0.0 onwards + character output for
the input of -NaN with the floating-point conversions, e.g. { printf
"%e", "-nan" }' produces "+nan" rather than "-nan",
- for released gawk versions from 5.0.0 onwards + character output for
the input of Inf or NaN in the absence of the + or space flags with
the floating-point conversions, e.g. { printf "%e", "inf" }' produces
"+inf" rather than "inf",
- for released gawk versions of up to 4.2.1 missing + character for the
input of Inf or NaN with the + flag and the floating-point
conversions, e.g. { printf "%+e", "inf" }' produces "inf" rather than
"+inf",
- for released gawk versions of up to 4.2.1 missing space character for
the input of Inf or NaN with the space flag and the floating-point
conversions, e.g. { printf "% e", "nan" }' produces "nan" rather than
" nan",
- for released gawk versions from 5.0.0 onwards + character output for
the input of Inf or NaN with the space flag and the floating-point
conversions, e.g. { printf "% e", "inf" }' produces "+inf" rather than
" inf",
- for released gawk versions from 5.0.0 onwards the field width is
ignored for the input of Inf or NaN and the floating-point
conversions, e.g. { printf "%20e", "-inf" }' produces "-inf" rather
than " -inf",
NB for released gawk versions of up to 4.2.1 floating-point conversion
issues apply to the bignum mode only, as in the non-bignum mode system
sprintf(3) is used. As from version 5.0.0 specialized handling has been
added for [-]Inf and [-]NaN inputs and the issues listed apply to both
modes. The '--posix' flag makes gawk versions from 5.0.0 onwards avoid
the issue with field width and the + character unconditionally output
for the input of Inf or NaN, however not the remaining issues and then
the 'gensub' function is not supported in the POSIX mode, so to go this
path I deemed not worth it.
Each test completes within single seconds except for the long double
one. There the F/f formats produce a large number of digits, which
appears to be computationally intensive and CPU-bound. Standalone
execution time for 'tst-printf-format-p-ldouble --direct f' is in the
range of 00m36s for POWER9@2.166GHz and 09m52s for FU740@1.2GHz and
output redirected locally to /dev/null, and 10m11s for FU740 and output
redirected over 100Mbps network via SSH to /dev/null, so the throughput
of the network adds very little (~3.2% in this case) to the processing
time. This is with IEEE 754 quad.
So I have scaled the timeout for 'tst-printf-format-skeleton-ldouble'
accordingly. Regardless, following recent practice the test has been
added to the standard rather than extended set. However, unlike most
of the remaining tests it has been split by the conversion specifier,
so as to allow better parallelization of this long-running test. As
a side effect this lets the test report the unsupported status for the
F/f conversions where applicable, so 'tst-printf-format-p-double' has
been split for consistency as well.
Only printf itself is handled at the moment, but the infrastructure
provides for all the printf family functions to be verified, changes
for which to be supplied separately. The complication around having
some tests iterating over all the relevant conversion specifiers and
other verifying conversion specifiers individually combined with
iterating over printf family functions has hit a peculiarity in GNU
make where the use of multiple targets with a pattern rule is handled
differently from such use with an ordinary rule. Consequently it
seems impossible to bulk-define a pattern rule using '$(foreach ...)',
where each target would simply trigger the recipe according to the
pattern and matching dependencies individually (such a rule does work,
but implies all targets to be updated with a single recipe execution).
Therefore as a compromise a single single-target pattern rule has been
defined that has listed all the conversion-specific scripts and all the
test executables as dependencies. Consequently tests will be rerun in
the absence of changes to their actual sources or scripts whenever an
unrelated file has changed that has been listed. Also all the formatted
printf output tests will always be built whenever any single one is to
be run. This only affects test development and not test runs in the
field, though it does change the order of execution of the individual
steps and also acts as a Makefile barrier in parallel runs. As the
execution time dominates the compilation time for these tests it is not
seen as a serious shortcoming.
As pointed out by Florian Weimer <fweimer@redhat.com> the malloc tracing
facility can take a substantial amount of time in calling dladdr(3) to
determine the caller's location. This is not needed by the verification
made with these tests, so I chose to interpose the symbol with a stub
implementation that always fails in the shared skeleton. We have total
control over the test environment, so I think it is a safe and minimal
impact approach. If there's ever anything else added to the tests that
would actually rely on dladdr(3) returning usable results, only then we
can think of a different approach.
Reviewed-by: DJ Delorie <dj@redhat.com>
The scanf family of functions like sscanf and fscanf currently
ignore nan() and nan(n-char-sequence). This happens because
__vfscanf_internal only checks for 'nan'.
This commit adds support for all valid nan types i.e. nan, nan()
and nan(n-char-sequence), where n-char-sequence can be
[a-zA-Z0-9_]+, thus fixing the bug 30647. Any other representation
of NaN should result in conversion error.
New tests are also added to verify the correct parsing of NaN types for
float, double and long double formats.
Signed-off-by: Avinal Kumar <avinal.xlvii@gmail.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Add tests of freopen adding or removing "c" (non-cancelling I/O) from
the mode string (so completing my planned tests of freopen with
different features used in the mode strings). Note that it's in the
nature of the uncertain time at which cancellation might act (possibly
during freopen, possibly during subsequent reads) that these can leak
memory or file descriptors, so these do not include leak tests.
Tested for x86_64.
The -Wp does not work properly if the compiler is configured to enable
fortify by default, since it bypasses the compiler driver (which defines
the fortify flags in this case).
This patch is similar to the one used on Ubuntu [1].
I checked with a build for x86_64-linux-gnu, i686-linux-gnu,
aarch64-linux-gnu, s390x-linux-gnu, and riscv64-linux-gnu with
gcc-13 that enables the fortify by default.
Co-authored-by: Matthias Klose <matthias.klose@canonical.com>
[1] https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/glibc/tree/debian/patches/ubuntu/fix-fortify-source.patch
Reviewed-by: DJ Delorie <dj@redhat.com>
Exercises fwrite's internal buffer when doing a file operation.
The new test, exercises 2 overflow behaviors:
1. Call fwrite multiple times making usage of fwrite's internal buffer.
The total number of bytes written is larger than fwrite's internal
buffer, forcing an automatic flush.
2. Call fwrite a single time with an amount of data that is larger than
fwrite's internal buffer.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Add tests of special cases for freopen that were omitted from the more
general tests of different modes and similar issues. The special
cases in the three tests here are logically unconnected, it was simply
convenient to put these tests in one patch.
* Test freopen with a NULL path to the new file, in a chroot. Rather
than asserting that this fails (logically, failure in this case is
an implementation detail; it's not required for freopen to rely on
/proc), verify that either it fails (without memory leaks) or that
it succeeds and behaves as expected on success. There is no check
for file descriptor leaks because the machinery for that also
depends on /proc, so can't be used in a chroot.
* Test that freopen and freopen64 are genuinely different in
configurations with 32-bit off_t by checking for an EFBIG trying to
write past 2GB in a file opened with freopen in such a configuration
but no error with 64-bit off_t or when opening with freopen64.
* Test freopen of stdin, stdout and stderr.
Tested for x86_64 and x86.
As reported in bug 23675 and shown up in the recently added tests of
different cases of freopen (relevant part of the test currently
conditioned under #if 0 to avoid a failure resulting from this bug),
freopen wrongly forces the stream to unoriented even when a mode with
,ccs= is specified, though such a mode is supposed to result in a
wide-oriented stream. Move the clearing of _mode to before the actual
reopening occurs, so that the main fopen implementation can leave a
wide-oriented stream in the ,ccs= case.
Tested for x86_64.
As reported in bug 32140, freopen leaks the FILE object when it
returns NULL: there is no valid use of the FILE * pointer (including
passing to freopen again or to fclose) after such an error return, so
the underlying object should be freed. Add code to free it.
Note 1: while I think it's clear from the relevant standards that the
object should be freed and the FILE * can't be used after the call in
this case (the stream is closed, which ends the lifetime of the FILE),
it's entirely possible that some existing code does in fact try to use
the existing FILE * in some way and could be broken by this change.
(Though the most common case for freopen may be stdin / stdout /
stderr, which _IO_deallocate_file explicitly checks for and does not
deallocate.)
Note 2: the deallocation is only done in the _IO_IS_FILEBUF case.
Other kinds of streams bypass all the freopen logic handling closing
the file, meaning a call to _IO_deallocate_file would neither be safe
(the FILE might still be linked into the list of all open FILEs) nor
sufficient (other internal memory allocations associated with the file
would not have been freed). I think the validity of freopen for any
other kind of stream will need clarifying with the Austin Group, but
if it is valid in any such case (where "valid" means "not undefined
behavior so required to close the stream" rather than "required to
successfully associate the stream with the new file in cases where
fopen would work"), more significant changes would be needed to ensure
the stream gets fully closed.
Tested for x86_64.
As reported in bug 32134, freopen does not clear the flags set in
fp->_flags2 by the "e", "m" or "c" mode characters. Clear these so
that they can be set or not as appropriate from the mode string passed
to freopen. The relevant test for "e" in tst-freopen2-main.c is
enabled accordingly; "c" is expected to be covered in a separately
written test (and while tst-freopen2-main.c does include transitions
to and from "m", that's not really a semantic flag intended to result
in behaving in an observably different way).
Tested for x86_64.
freopen is rather minimally tested in libio/tst-freopen and
libio/test-freopen. Add some more thorough tests, covering different
cases for change of mode in particular. The tests are run for both
freopen and freopen64 (given that those functions have two separate
copies of much of the code, so any bug fix directly in the freopen
code would probably need applying in both places).
Note that there are two parts of the tests disabled because of bugs
discovered through running the tests, with bug numbers given in
comments. I expect to address those separately. The tests also don't
cover changes to cancellation ("c" in mode); I think that will better
be handled through a separate test. Also to handle separately:
testing on stdin / stdout / stderr; documenting lack of support for
streams opened with popen / fmemopen / open_memstream / fopencookie;
maybe also a chroot test without /proc; maybe also more thorough tests
for large file handling on 32-bit systems (freopen64).
Tested for x86_64.
There is very little test coverage for getline (only a minimal
stdio-common/tstgetln.c which doesn't verify anything about the
results of the getline calls). Add some more thorough tests
(generally using fopencookie for convenience in testing various cases
for what the input and possible errors / EOF in the file read might
look like).
Note the following regarding testing of error cases:
* Nothing is said in the specifications about what if anything might
be written into the buffer, and whether it might be reallocated, in
error cases. The expectation of the tests (required to avoid memory
leaks on error) is that at least on error cases, the invariant that
lineptr points to at least n bytes is maintained.
* The optional EOVERFLOW error case specified in POSIX, "The number of
bytes to be written into the buffer, including the delimiter
character (if encountered), would exceed {SSIZE_MAX}.", doesn't seem
practically testable, as any case reading so many characters (half
the address space) would also be liable to run into allocation
failure along (ENOMEM) the way.
* If a read error occurs part way through reading an input line, it
seems unclear whether a partial line should be returned by getline
(avoid input getting lost), which is what glibc does at least in the
fopencookie case used in this test, or whether getline should return
-1 (error) (so avoiding the program misbehaving by processing a
truncated line as if it were complete). (There was a short,
inconclusive discussion about this on the Austin Group list on 9-10
November 2014.)
* The POSIX specification of getline inherits errors from fgetc. I
didn't try to cover fgetc errors systematically, just one example of
such an error.
Tested for x86_64 and x86.
Macros will automatically use the correct types, without
having to fiddle with internal glibc macros. It's also
impossible to get the types wrong due to aliasing because
support_check_stat_fd and support_check_stat_path do not
depend on the struct stat* types.
The changes reveal some inconsistencies in tests.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
If a file descriptor is left unclosed and is cleaned up by _IO_cleanup
on exit, its backup buffer remains unfreed, registering as a leak in
valgrind. This is not strictly an issue since (1) the program should
ideally be closing the stream once it's not in use and (2) the program
is about to exit anyway, so keeping the backup buffer around a wee bit
longer isn't a real problem. Free it anyway to keep valgrind happy
when the streams in question are the standard ones, i.e. stdout, stdin
or stderr.
Also, the _IO_have_backup macro checks for _IO_save_base,
which is a roundabout way to check for a backup buffer instead of
directly looking for _IO_backup_base. The roundabout check breaks when
the main get area has not been used and user pushes a char into the
backup buffer with ungetc. Fix this to use the _IO_backup_base
directly.
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
When ungetc is called on an unused stream, the backup buffer is
allocated without the main get area being present. This results in
every subsequent ungetc (as the stream remains in the backup area)
checking uninitialized memory in the backup buffer when trying to put a
character back into the stream.
Avoid comparing the input character with buffer contents when in backup
to avoid this uninitialized read. The uninitialized read is harmless in
this context since the location is promptly overwritten with the input
character, thus fulfilling ungetc functionality.
Also adjust wording in the manual to drop the paragraph that says glibc
cannot do multiple ungetc back to back since with this change, ungetc
can actually do this.
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Complement commit b03e4d7bd2 ("stdio: fix vfscanf with matches longer
than INT_MAX (bug 27650)") and add a test case for the issue, inspired
by the reproducer provided with the bug report.
This has been verified to succeed as from the commit referred and fail
beforehand.
As the test requires 2GiB of data to be passed around its performance
has been evaluated using a choice of systems and the execution time
determined to be respectively in the range of 9s for POWER9@2.166GHz,
24s for FU740@1.2GHz, and 40s for 74Kf@950MHz. As this is on the verge
of and beyond the default timeout it has been increased by the factor of
8. Regardless, following recent practice the test has been added to the
standard rather than extended set.
Reviewed-by: DJ Delorie <dj@redhat.com>
The conditionals for several mtrace-based tests in catgets, elf, libio,
malloc, misc, nptl, posix, and stdio-common were incorrect leading to
test failures when bootstrapping glibc without perl.
The correct conditional for mtrace-based tests requires three checks:
first checking for run-built-tests, then build-shared, and lastly that
PERL is not equal to "no" (missing perl).
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This reverts commit 30a745450e.
On ppc64le, without <stdio.h>, vfscanf is used and with <stdio.h>
__isoc23_vfscanfieee128 is used. I am reverting this since it doesn't
work on all targets.
Add a test for fscanf of long double without including <stdio.h>.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Sunil K Pandey <skpgkp2@gmail.com>
The gnulib version contains an important change (9ce573cde), which
fixes some problems with multithreading, entropy loss, and ASLR leak
nfo. It also fixes an issue where getrandom is not being used
on some new files generation (only for __GT_NOCREATE on first try).
The 044bf893ac removed __path_search, which is now moved to another
gnulib shared files (stdio-common/tmpdir.{c,h}). Tthis patch
also fixes direxists to use __stat64_time64 instead of __xstat64,
and move the include of pathmax.h for !_LIBC (since it is not used
by glibc). The license is also changed from GPL 3.0 to 2.1, with
permission from the authors (Bruno Haible and Paul Eggert).
The sync also removed the clock fallback, since clock_gettime
with CLOCK_REALTIME is expected to always succeed.
It syncs with gnulib commit 323834962817af7b115187e8c9a833437f8d20ec.
Checked on x86_64-linux-gnu.
Co-authored-by: Bruno Haible <bruno@clisp.org>
Co-authored-by: Paul Eggert <eggert@cs.ucla.edu>
Reviewed-by: Bruno Haible <bruno@clisp.org>
Complete the internal renaming from "C2X" and related names in GCC by
renaming *-c2x and *-gnu2x tests to *-c23 and *-gnu23.
Tested for x86_64, and with build-many-glibcs.py for powerpc64le.
WG14 decided to use the name C23 as the informal name of the next
revision of the C standard (notwithstanding the publication date in
2024). Update references to C2X in glibc to use the C23 name.
This is intended to update everything *except* where it involves
renaming files (the changes involving renaming tests are intended to
be done separately). In the case of the _ISOC2X_SOURCE feature test
macro - the only user-visible interface involved - support for that
macro is kept for backwards compatibility, while adding
_ISOC23_SOURCE.
Tested for x86_64.
The strlen might trigger and invalid GOT entry if it used before
the process is self-relocated (for instance on dl-tunables if any
error occurs).
For i386, _dl_writev with PIE requires to use the old 'int $0x80'
syscall mode because the calling the TLS register (gs) is not yet
initialized.
Checked on x86_64-linux-gnu.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
All the crypt related functions, cryptographic algorithms, and
make requirements are removed, with only the exception of md5
implementation which is moved to locale folder since it is
required by localedef for integrity protection (libc's
locale-reading code does not check these, but localedef does
generate them).
Besides thec code itself, both internal documentation and the
manual is also adjusted. This allows to remove both --enable-crypt
and --enable-nss-crypt configure options.
Checked with a build for all affected ABIs.
Co-authored-by: Zack Weinberg <zack@owlfolio.org>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
ISO C2x defines scanf length modifiers wN (for intN_t / int_leastN_t /
uintN_t / uint_leastN_t) and wfN (for int_fastN_t / uint_fastN_t).
Add support for those length modifiers, similar to the printf support
previously added.
Tested for x86_64 and x86.
Avoid potential stack overflow from unbounded alloca. Use the existing
scratch_buffer instead.
Add testcases to exercise the code as suggested by Adhemerval Zanella Netto.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Some locales define a list of mapping pairs of alternate digits and
separators for input digits (to_inpunct). This require the scanf
to create a list of all possible inputs for the optional type
modifier 'I'.
Checked on x86_64-linux-gnu.
Reviewed-by: Joe Simmons-Talbott <josimmon@redhat.com>
Since the _FORTIFY_SOURCE feature uses some routines of Glibc, they need to
be excluded from the fortification.
On top of that:
- some tests explicitly verify that some level of fortification works
appropriately, we therefore shouldn't modify the level set for them.
- some objects need to be build with optimization disabled, which
prevents _FORTIFY_SOURCE to be used for them.
Assembler files that implement architecture specific versions of the
fortified routines were not excluded from _FORTIFY_SOURCE as there is no
C header included that would impact their behavior.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Few tests using swprintf are passing incorrect maxlen parameter.
This triggers an abort when _FORTIFY_SOURCE is enabled.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
ISO C2x defines scanf %b for input of binary integers (with an
optional 0b or 0B prefix). Implement such support, along with the
corresponding SCNb* macros in <inttypes.h>. Unlike the support for
binary integers with 0b or 0B prefix with scanf %i, this is supported
in all versions of scanf (independent of the standards mode used for
compilation), because there are no backwards compatibility concerns
(%b wasn't previously a supported format) the way there were for %i.
Tested for x86_64 and x86.
ISO C2x defines printf length modifiers wN (for intN_t / int_leastN_t
/ uintN_t / uint_leastN_t) and wfN (for int_fastN_t / uint_fastN_t).
Add support for those length modifiers (such a feature was previously
requested in bug 24466). scanf support is to be added separately.
GCC 13 has format checking support for these modifiers.
When used with the support for registering format specifiers, these
modifiers are translated to existing flags in struct printf_info,
rather than trying to add some way of distinguishing them without
breaking the printf_info ABI. C2x requires an error to be returned
for unsupported values of N; this is implemented for printf-family
functions, but the parse_printf_format interface doesn't support error
returns, so such an error gets discarded by that function.
Tested for x86_64 and x86.
With fortification enabled, fgets calls return result needs to be checked,
has it gets the __wur macro enabled.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
With fortification enabled, fread calls return result needs to be checked,
has it gets the __wur macro enabled.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
When enabling _FORTIFY_SOURCE, some functions now lead to warnings when
their result is not checked.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
With fortification enabled, few function calls return result need to be
checked, has they get the __wur macro enabled.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
With fortification enabled, ftruncate calls return result needs to be
checked, has it gets the __wur macro enabled.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Sort tests against updated scripts/sort-makefile-lines.py.
No changes in generated code.
No regressions on x86_64 and i686.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reflow Makefile.
Sort using scripts/sort-makefile-lines.py.
Code generation is changed as routines are linked in sorted order
as expected.
No regressions on x86_64 and i686.
In this case, _itoa_word () is already defined inline in the header (see
sysdeps/generic/_itoa.h), and the second definition causes an error.
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
Message-Id: <20230319151017.531737-11-bugaevc@gmail.com>
Instead of using a special ELF section along with a linker script
directive to put the IO vtables within the RELRO section, the libio
vtables are all moved to an array marked as data.relro (so linker
will place in the RELRO segment without the need of extra directives).
To avoid static linking namespace issues and including all vtable
referenced objects, all required function pointers are set to weak alias.
Checked on x86_64-linux-gnu, i686-linux-gnu, and aarch64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Instead define the required fields in system dependend files. The only
system dependent definition is FILENAME_MAX, which should match POSIX
PATH_MAX, and it is obtained from either kernel UAPI or mach headers.
Currently set pre-defined value from current kernels.
It avoids a circular dependendy when including stdio.h in
gen-as-const-headers files.
Checked on x86_64-linux-gnu and i686-linux-gnu
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
They are both used by __libc_freeres to free all library malloc
allocated resources to help tooling like mtrace or valgrind with
memory leak tracking.
The current scheme uses assembly markers and linker script entries
to consolidate the free routine function pointers in the RELRO segment
and to be freed buffers in BSS.
This patch changes it to use specific free functions for
libc_freeres_ptrs buffers and call the function pointer array directly
with call_function_static_weak.
It allows the removal of both the internal macros and the linker
script sections.
Checked on x86_64-linux-gnu, i686-linux-gnu, and aarch64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Exactly the same as 35bcb08eaa.
If using -D_FORITFY_SOURCE=3 (in my case, I've patched GCC to add
=3 instead of =2 (we've done =2 for years in Gentoo)), building
glibc tests will fail on tst-bz11319-fortify2 like:
```
<command-line>: error: "_FORTIFY_SOURCE" redefined [-Werror]
<built-in>: note: this is the location of the previous definition
cc1: all warnings being treated as errors
```
It's just because we're always setting -D_FORTIFY_SOURCE=2
rather than unsetting it first. If F_S is already 2, it's harmless,
but if it's another value (say, 1, or 3), the compiler will bawk.
(I'm not aware of a reason this couldn't be tested with =3,
but the toolchain support is limited for that (too new), and we want
to run the tests everywhere possible.)
As Siddhesh noted previously, we could implement some fallback
logic to determine the maximal F_S value supported by the toolchain,
which is a bit easier now that autoconf-archive has been updated for F_S=3
(https://github.com/autoconf-archive/autoconf-archive/pull/269), but let's
revisit this if it continues to crop up.
Signed-off-by: Sam James <sam@gentoo.org>
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
WG14 recently accepted two additions to the printf/scanf %b/%B
support: there are now PRIb* and SCNb* macros in <inttypes.h>, and
printf %B is now an optional feature defined in normative text,
instead of recommended practice, with corresponding PRIB* macros that
can also be used to test whether that optional feature is supported.
See N3072 items 14 and 15 for details (those changes were accepted,
some other changes in that paper weren't).
Add the corresponding PRI* macros to glibc and update one place in the
manual referring to %B as recommended. (SCNb* should naturally be
added at the same time as the corresponding scanf %b support.)
Tested for x86_64 and x86.
C2x adds binary integer constants starting with 0b or 0B, and supports
those constants for the %i scanf format (in addition to the %b format,
which isn't yet implemented for scanf in glibc). Implement that scanf
support for glibc.
As with the strtol support, this is incompatible with previous C
standard versions, in that such an input string starting with 0b or 0B
was previously required to be parsed as 0 (with the rest of the input
potentially matching subsequent parts of the scanf format string).
Thus this patch adds 12 new __isoc23_* functions per long double
format (12, 24 or 36 depending on how many long double formats the
glibc configuration supports), with appropriate header redirection
support (generally very closely following that for the __isoc99_*
scanf functions - note that __GLIBC_USE (DEPRECATED_SCANF) takes
precedence over __GLIBC_USE (C2X_STRTOL), so the case of GNU
extensions to C89 continues to get old-style GNU %a and does not get
this new feature). The function names would remain as __isoc23_* even
if C2x ends up published in 2024 rather than 2023.
When scanf %b support is added, I think it will be appropriate for all
versions of scanf to follow C2x rules for inputs to the %b format
(given that there are no compatibility concerns for a new format).
Tested for x86_64 (full glibc testsuite). The first version was also
tested for powerpc (32-bit) and powerpc64le (stdio-common/ and wcsmbs/
tests), and with build-many-glibcs.py.
This is a partial fix for mishandling of grouping when formatting
integers. It properly computes the width in the presence of grouping
characters when the width is larger than the number of significant
digits. The precision related issue is documented in bug 23432.
Co-authored-by: Andreas Schwab <schwab@suse.de>
The __printf_buffer_flush_dprintf function needs to record that
the buffer has been written before reusing it. Without this
accounting, dprintf always returns zero.
Fixes commit 8ece45e4f5
("libio: Convert __vdprintf_internal to buffers").
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Always null-terminate the buffer and set E2BIG if the buffer is too
small. This fixes bug 27857.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The internal buffer size is set to 2048 bytes. This is less than
the original BUFSIZ value used by buffered_vfprintf before
the conversion, but it hopefully covers all cases where write
boundaries matter.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The buffer resizing algorithm is slightly different. The initial
buffer is on the stack, and small buffers are directly allocated
on the heap using the exact required size. The overhead of the
additional copy is compensated by the lowered setup cost for buffers
compared to libio streams.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Joseph Myers
Tulio Magno Quites Machado Filho
Martin Coufal
Xi Ruoyao
Adhemerval Zanella
Paul Eggert
Sam James
Dmitry Chestnykh
H.J. Lu
Siddhesh Poyarekar
Alejandro Colomar
Maciej W. Rozycki
Avinal Kumar
Florian Weimer
Sergey Kolosov
DJ Delorie
Carlos O'Donell
Mike FABIAN
Joseph Myers
Joe Simmons-Talbott
Frédéric Bérat
Paul Pluzhnikov
Sergey Bugaev
Andreas Schwab