Some of the new multi-arch string functions for x86-64 were
not aligned to 16 byte boundarie,s possibly creating unnecessary
cache line misses and delays.
This patch adds SSSE3 strcpy/stpcpy. I got up to 4X speed up on Core 2
and Core i7. I disabled it on Atom since SSSE3 version is slower for
shorter (<64byte) data.
The test to call the indirect function now includes a subtest to
checked whether the symbol is defined. When coming to that point
this is almost always the case. The test for STT_GNU_IFUNC on the
other hand rarely is true. Move it to the front means we don't have
to perform the second test unless really necessary.
SO far Intel and AMD use exactly the same bits meaning the same
things in CPUID index 1. Simplify the code. Should an architecture
come along which doesn't use the same semantics then it must use a
different index value than COMMON_CPUID_INDEX_1.
If longjmp restores the stack frame to an address which is beyond
the stack frame at the time of the longjmp call it would install
an uninitialized stack frame. If compiled with _FORTIFY_SOURCE
defined, longjmp will now bail out in this situation.
from definition.
* sysdeps/x86_64/dl-machine.h (elf_machine_rela): Don't define
label if it is not used.
* elf/dl-profile.c (_dl_start_profile): Define real-type variant
of gmon_hist_hdr and gmon_hdr structures and use them.
* elf/dl-load.c (open_verify): Add temporary variable to avoid
warning.
* nscd/nscd_helper.c (get_mapping): Avoid casts to avoid warnings.
* sunrpc/clnt_raw.c (clntraw_private_s): Use union in definition
to avoid cast.
* inet/rexec.c (rexec_af): Make sa2 a union to avoid warnings.
* inet/rcmd.c (rcmd_af): Make from a union of the various needed types
to avoid warnings.
(iruserok_af): Use ss_family instead of casts.
* gmon/gmon.c (write_hist): Define real-type variant of
gmon_hist_hdr structure and use it.
(write_gmon): Likewise for gmon_hdr.
* sysdeps/unix/sysv/linux/readv.c: Avoid declaration of replacement
function if we are not going to define it.
* sysdeps/unix/sysv/linux/writev.c: Likewise.
* inet/inet6_option.c (optin_alloc): Add temporary variable to
avoid warning.
* libio/strfile.h (struct _IO_streambuf): Use correct type and
name of VTable element.
* libio/iovsprintf.c: Avoid casts to avoid warnings.
* libio/iovsscanf.c: Likewise.
* libio/vasprintf.c: Likewise.
* libio/vsnprintf.c: Likewise.
* stdio-common/isoc99_vsscanf.c: Likewise.
* stdlib/strfmon_l.c: Likewise.
* debug/vasprintf_chk.c: Likewise.
* debug/vsnprintf_chk.c: Likewise.
* debug/vsprintf_chk.c: Likewise.
* sysdeps/x86_64/add_n.S: New file.
* sysdeps/x86_64/addmul_1.S: New file.
* sysdeps/x86_64/lshift.S: New file.
* sysdeps/x86_64/mul_1.S: New file.
* sysdeps/x86_64/rshift.S: New file.
* sysdeps/x86_64/sub_n.S: New file.
* sysdeps/x86_64/submul_1.S: New file.
* inet/inet6_rth.c (inet6_rth_add): Add some error checking.
Patch mostly by Yang Hongyang <yanghy@cn.fujitsu.com>.
* inet/Makefile (tests): Add tst-inet6_rth.
* inet/tst-inet6_rth.c: New file.
alignment of La_x86_64_regs. Store xmm parameters.
(reloc_index): Define.
(_dl_fixup): Rename reloc_offset parameter to reloc_arg.
(_dl_fixup_profile): Likewise. Use reloc_index instead of
computing index from reloc_offset.
(_dl_call_pltexit): Likewise.
* sysdeps/x86_64/dl-trampoline.S (_dl_runtime_resolve): Just pass
the relocation index to _dl_fixup.
(_dl_runtime_profile): Likewise for _dl_fixup_profile and
_dl_call_pltexit.
* sysdeps/x86_64/dl-runtime.c: New file.
* sysdeps/x86_64/dl-trampoline.S (_dl_runtime_profile): Fix
alignement of La_x86_64_regs. Store xmm parameters.
Patch mostly by Jiri Olsa <olsajiri@gmail.com>.
* nscd/mem.c (gc): Use alloca_count to get the real stack usage.
* include/alloca.h (alloca_account): Define.
* sysdeps/x86_64/stackinfo.h (stackinfo_get_sp): Define.
(stackinfo_sub_sp): Define.
2008-2-26 Harsha Jagasia <harsha.jagasia@amd.com>
* sysdeps/x86_64/cacheinfo.c (NOT_USED_RIGHT_NOW): Remove ifdef guards.
* sysdeps/x86_64/memset.S: Rewrite non-SSE code path as tuned for AMD
Barcelona machine. Make default fall through branch of
__x86_64_preferred_memory_instruction check as the integer code path.
2007-10-15 H.J. Lu <hongjiu.lu@intel.com>
* sysdeps/x86_64/cacheinfo.c
(__x86_64_preferred_memory_instruction): New variable.
(init_cacheinfo): Initialize __x86_64_preferred_memory_instruction.
* sysdeps/x86_64/memset.S: Rewrite.
2008-01-08 Jakub Jelinek <jakub@redhat.com>
* malloc/malloc.c (public_cALLOc): For arenas other than
2007-10-16 Ulrich Drepper <drepper@redhat.com>
* time/tzfile.c (__tzfile_read): Take extra memory requested by caller
into account when copying TZ string.
a local label rather than HIDDEN_JUMPTARGET.
2007-10-16 Jakub Jelinek <jakub@redhat.com>
* sysdeps/x86_64/memset.S: Jump from bzero to memset using
a local label rather than HIDDEN_JUMPTARGET.
from __x86_64_core_cache_size_half.
(init_cacheinfo): Compute shared cache size for AMD processors with
shared L3 correctly.
* sysdeps/x86_64/memcpy.S: Adjust for __x86_64_data_cache_size_half
name change.
Patch in large parts by Evandro Menezes.
* misc/bits/syslog.h (syslog): When __va_arg_pack is defined,
implement as __extern_always_inline function.
(vsyslog): Define as __extern_always_inline function unconditionally.
* libio/bits/stdio2.h (sprintf, snprintf, printf, fprintf):
When __va_arg_pack is defined, implement as __extern_always_inline
functions.
(vsprintf, vsnprintf, vprintf, vfprintf): Define as
__extern_always_inline functions unconditionally.
* libio/bits/stdio.h (vprintf): Ifdef out the inline when
bits/stdio2.h will be included.
* wcsmbs/bits/wchar2.h (__swprintf_alias): New redirect.
(swprintf, wprintf, fwprintf): When __va_arg_pack is defined,
implement as __extern_always_inline functions.
(vswprintf, vwprintf, vfwprintf): Define as
__extern_always_inline functions unconditionally.
* debug/tst-chk1.c (do_test): Enable remaining tests for C++.
2007-09-03 Jakub Jelinek <jakub@redhat.com>
* misc/sys/cdefs.h (__extern_inline, __extern_always_inline): Only
define in C++ for GCC 4.3+, in C++ always use __gnu_inline__
attribute.
* include/features.h (__USE_EXTERN_INLINES): Define only when
__extern_inline is defined.
* stdlib/stdlib.h: Include bits/stdlib.h when __extern_always_inline
is defined instead of when not __cplusplus.
* misc/sys/syslog.h: Include bits/syslog.h when __extern_always_inline
is defined instead of when not __cplusplus.
* socket/sys/socket.h: Include bits/socket2.h when
__extern_always_inline is defined instead of when not __cplusplus.
* libio/stdio.h: Include bits/stdio2.h when __extern_always_inline
is defined instead of when not __cplusplus.
* posix/unistd.h: Include bits/unistd.h when __extern_always_inline
is defined instead of when not __cplusplus.
* string/string.h: Include bits/string3.h when __extern_always_inline
is defined instead of when not __cplusplus.
* wcsmbs/wchar.h: Include bits/wchar2.h when __extern_always_inline
is defined instead of when not __cplusplus.
(btowc, wctob): Don't guard the inlines with ifndef __cplusplus.
* io/fcntl.h: Don't include bits/fcntl2.h if __extern_always_inline
is not defined.
* misc/bits/syslog-ldbl.h: Guard *_chk stuff with
defined __extern_always_inline instead of !defined __cplusplus.
* libio/bits/stdio-ldbl.h: Likewise.
* wcsmbs/bits/wchar-ldbl.h: Likewise.
* misc/bits/syslog.h (syslog): Don't define for C++.
(vsyslog): Use __extern_always_inline function for C++ instead of
a macro.
* libio/bits/stdio.h (__STDIO_INLINE): Define to __extern_inline
whenever that macro is defined.
(vprintf): Don't provide the inline for C++.
(fread_unlocked, fwrite_unlocked): Don't define the macros for C++.
* libio/bits/stdio2.h (sprintf, snprintf, printf, fprintf): Don't
define the macros for C++.
(vsprintf, vsnprintf, vprintf, vfprintf): Define as
__extern_always_inline functions for C++.
* io/sys/stat.h (stat, lstat, fstat, fstatat, mknod, mknodat,
stat64, lstat64, fstat64, fstatat64): Don't define if not
__USE_EXTERN_INLINES.
* wcsmbs/bits/wchar2.h: Fix #error message.
(swprintf, wprintf, fwprintf): Don't define the macros for C++.
(vswprintf, vwprintf, vfwprintf): Define using
__extern_always_inline functions for C++.
* string/bits/string3.h: Don't #undef macros if __cplusplus.
(memcpy, memmove, mempcpy, memset, bcopy, bzero, strcpy, stpcpy,
strncpy, strcat, strncat): Define as __extern_always_inline
functions instead of macros for C++.
* math/bits/cmathcalls.h: Guard __extern_inline routines with
defined __extern_inline.
* sysdeps/alpha/fpu/bits/mathinline.h (__MATH_INLINE): Define
to __extern_inline whenever that macro is defined.
* sysdeps/ia64/fpu/bits/mathinline.h (__MATH_INLINE): Likewise.
* sysdeps/i386/fpu/bits/mathinline.h (__MATH_INLINE): Likewise.
* sysdeps/i386/i486/bits/string.h (__STRING_INLINE): Likewise.
* sysdeps/s390/bits/string.h (__STRING_INLINE): Likewise.
* sysdeps/s390/fpu/bits/mathinline.h (__MATH_INLINE): Likewise.
* sysdeps/powerpc/fpu/bits/mathinline.h (__MATH_INLINE): Likewise.
* sysdeps/x86_64/fpu/bits/mathinline.h (__MATH_INLINE): Likewise.
* sysdeps/sparc/fpu/bits/mathinline.h (__MATH_INLINE): Likewise.
* sysdeps/unix/sysv/linux/sys/sysmacros.h (gnu_dev_major,
gnu_dev_minor, gnu_dev_makedev): Remove __extern_inline from
prototypes. Only provide __extern_inline routines if
__USE_EXTERN_INLINES.
* debug/Makefile: Add rules to build and run tst-{,lfs}chk{4,5,6}
tests.
* debug/tst-chk1.c (do_prepare, do_test): Allow compilation as C++.
For now avoid some *printf tests in C++. Skip all testing
if __USE_FORTIFY_LEVEL is defined, but __extern_always_inline macro
is not.
* debug/tst-chk4.cc: New file.
* debug/tst-chk5.cc: New file.
* debug/tst-chk6.cc: New file.
* debug/tst-lfschk4.cc: New file.
* debug/tst-lfschk5.cc: New file.
* debug/tst-lfschk6.cc: New file.
* include/wchar.h (__vfwprintf_chk, __vswprintf_chk): Avoid
prototypes in C++.
* include/stdio.h (__sprintf_chk, __snprintf_chk, __vsprintf_chk,
__vsnprintf_chk, __printf_chk, __fprintf_chk, __vprintf_chk,
__vfprintf_chk, __fgets_unlocked_chk, __fgets_chk): Likewise.
* sysdeps/x86_64/fpu/k_cosl.c: New file.
* sysdeps/x86_64/fpu/k_rem_pio2l.c: New file.
* sysdeps/x86_64/fpu/k_sinl.c: New file.
* sysdeps/x86_64/fpu/k_tanl.c: New file.
2007-03-27 Jakub Jelinek <jakub@redhat.com>
[BZ #3306]
* math/math_private.h (math_opt_barrier, math_force_eval): Define.
* sysdeps/i386/fpu/math_private.h: New file.
* sysdeps/x86_64/fpu/math_private.h: New file.
* math/s_nexttowardf.c (__nexttowardf): Use math_opt_barrier and
math_force_eval macros. Use "+m" constraint on asm rather than
"=m" and "m".
* math/s_nextafter.c (__nextafter): Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_nexttoward.c (__nexttoward):
Likewise.
* sysdeps/ieee754/flt-32/s_nextafterf.c (__nextafterf): Likewise.
* sysdeps/ieee754/ldbl-128/s_nexttoward.c (__nexttoward): Likewise.
* sysdeps/ieee754/ldbl-96/s_nexttoward.c (__nexttoward): Likewise.
* sysdeps/i386/fpu/s_nextafterl.c (__nextafterl): Use
math_opt_barrier and math_force_eval macros.
* sysdeps/ieee754/ldbl-128/s_nextafterl.c (__nextafterl): Likewise.
* sysdeps/ieee754/ldbl-96/s_nextafterl.c (__nextafterl): Likewise.
* sysdeps/i386/fpu/s_nexttoward.c: Include float.h.
(__nexttoward): Use math_opt_barrier and
math_force_eval macros. Use "+m" constraint on asm rather than
"=m" and "m". Only use asm to force double result if
FLT_EVAL_METHOD is 2.
* sysdeps/i386/fpu/s_nexttowardf.c: Include float.h.
(__nexttowardf): Use math_opt_barrier and
math_force_eval macros. Use "+m" constraint on asm rather than
"=m" and "m". Only use asm to force double result if
FLT_EVAL_METHOD is not 0.
* sysdeps/ieee754/ldbl-128ibm/s_nexttowardf.c: Include float.h.
(__nexttowardf): Use math_opt_barrier and
math_force_eval macros. If FLT_EVAL_METHOD is not 0, force
x to float using asm.
* sysdeps/ieee754/ldbl-opt/s_nexttowardfd.c: Include float.h.
(__nldbl_nexttowardf): Use math_opt_barrier and
math_force_eval macros. If FLT_EVAL_METHOD is not 0, force
x to float using asm.
* sysdeps/ieee754/ldbl-96/s_nexttowardf.c: Include float.h.
(__nexttowardf): Use math_opt_barrier and math_force_eval
macros. If FLT_EVAL_METHOD is not 0, force x to float using asm.
* math/bug-nextafter.c (zero, inf): New variables.
(main): Add new tests.
* math/bug-nexttoward.c (zero, inf): New variables.
(main): Add new tests.
2007-03-22 Jakub Jelinek <jakub@redhat.com>
[BZ #3427]
* sysdeps/x86_64/fpu/feholdexcpt.c (feholdexcept): Clear all
exceptions both in SW and MXCSR.
* sysdeps/x86_64/fpu/feupdateenv.c: New file.
* sysdeps/x86_64/fpu/feenablxcpt.c (feenableexcept): Remove dead code.
* sysdeps/x86_64/fpu/fedisblxcpt.c (fedisableexcept): Likewise.
* sysdeps/i386/fpu/feholdexcpt.c (feholdexcept): Clear all exceptions
in MXCSR if SSE is available.
* sysdeps/i386/fpu/feupdateenv.c: Include unistd.h, dl-procinfo.h
and ldsodefs.h.
(__feupdateenv): Query exceptions also from MXCSR if SSE is available.
Fix comment typo.
* sysdeps/ia64/fpu/feholdexcpt.c (feholdexcept): Clear all exceptions.
Return 0 rather than 1.
* sysdeps/ia64/fpu/feupdateenv.c (feupdateenv): Fix comment typo.
Remove incorrect part of a comment. Fix argument to feraiseexcept.
* math/test-fenv.c (feholdexcept_tests): New function.
(main): Call it.
2007-01-05 Richard B. Kreckel <kreckel@ginac.de>
[BZ #3427]
* sysdeps/i386/fpu/feholdexcpt.c (feholdexcept): Clear all exceptions
in SW.
2007-03-01 Jakub Jelinek <jakub@redhat.com>
[BZ #4096]
* sysdeps/ieee754/dbl-64/e_pow.c (__ieee754_pow): Check for NaN
earlier.
* math/libm-test.inc (pow_test): Add more tests involving NaNs.
* sysdeps/i386/fpu/e_powf.S (__ieee754_powf): Avoid invalid exception
for x qNaN and y either +-inf or non-integer value.
* sysdeps/i386/fpu/e_pow.S (__ieee754_pow): Likewise.
* sysdeps/i386/fpu/e_powl.S (__ieee754_powl): Likewise.
* sysdeps/x86_64/fpu/e_powl.S (__ieee754_powl): Likewise.
2007-02-16 Ulrich Drepper <drepper@redhat.com>
[BZ #3851]
* locales/bn_BD: Fix full name of January and February.
Patch by Jamil Ahmed <jamil@bengalinux.org>.
to the list of i486+ CPUs.
* sysdeps/x86_64/bits/byteswap.h (__bswap_32): Likewise.
2007-02-12 Jakub Jelinek <jakub@redhat.com>
* sysdeps/i386/bits/byteswap.h (__bswap_32): Add __amdfam10__
to the list of i486+ CPUs.
* sysdeps/x86_64/bits/byteswap.h (__bswap_32): Likewise.
atomic_compare_and_exchange_val_acq,
atomic_compare_and_exchange_bool_acq, and atomic_exchange_and_add
using __sync_* built-ins for gcc >= 4.1.
* sysdeps/x86_64/bits/atomic.h: Likewise.
2006-10-17 Jakub Jelinek <jakub@redhat.com>
* sunrpc/xdr_mem.c (xdrmem_setpos): Don't compare addresses
as signed longs, check for x_base + pos overflow.
* sunrpc/Makefile (tests): Add tst-xdrmem2.
* sunrpc/tst-xdrmem2.c: New test.
2006-10-18 Ulrich Drepper <drepper@redhat.com>
* elf/dl-lookup.c (_dl_lookup_symbol_x): Add warning to
_dl_lookup_symbol_x code.
2006-10-17 Jakub Jelinek <jakub@redhat.com>
* elf/dl-runtime.c: Include sysdep-cancel.h.
(_dl_fixup, _dl_profile_fixup): Use __rtld_mrlock_* and
scoperec->nusers only if !SINGLE_THREAD_P. Use atomic_*
instead of catomic_* macros.
* elf/dl-sym.c: Include sysdep-cancel.h.
(do_sym): Use __rtld_mrlock_* and scoperec->nusers only
if !SINGLE_THREAD_P. Use atomic_* instead of catomic_* macros.
* elf/dl-close.c: Include sysdep-cancel.h.
(_dl_close): Use __rtld_mrlock_* and scoperec->nusers only
if !SINGLE_THREAD_P. Use atomic_* instead of catomic_* macros.
* elf/dl-open.c: Include sysdep-cancel.h.
(dl_open_worker): Use __rtld_mrlock_* and scoperec->nusers only
if !SINGLE_THREAD_P. Use atomic_* instead of catomic_* macros.
2006-10-17 Jakub Jelinek <jakub@redhat.com>
[BZ #3313]
* malloc/malloc.c (malloc_consolidate): Set maxfb to address of last
fastbin rather than end of fastbin array.
2006-10-18 Ulrich Drepper <drepper@redhat.com>
* sysdeps/i386/i486/bits/atomic.h (catomic_decrement): Use correct
body macro.
* sysdeps/x86_64/bits/atomic.h
(__arch_c_compare_and_exchange_val_64_acq): Add missing casts.
(catomic_decrement): Use correct body macro.
2006-10-17 Jakub Jelinek <jakub@redhat.com>
* include/atomic.h: Add a unique prefix to all local variables
in macros.
* csu/tst-atomic.c (do_test): Test also catomic_* macros.
* include/link.h: Include <rtld-lowlevel.h>. Define struct
Ulrich Drepper
H.J. Lu
Jakub Jelinek
Roland McGrath