The regex engine tries to optimise Kleene star by avoiding backtracking
when it can detect that star’s operand cannot match what follows it in
the pattern.
For example, when ‘[[:alpha:]]*1’ tries to match a ‘foo’, the engine
will test the longest match for ‘[[:alpha:]]*’, namely ’foo’ which is
the entire string. Literal digit one still present in the pattern will
however not match the remaining empty string.
Normally, backtracking would be performed trying a shorter match for the
character class (namely ‘fo’ leaving ‘o’ in the string), but since the
engine knows whatever would be put back into the string cannot possibly
match literal digit one so no backtracking will be attempted.
In the regexes of the form ‘[[:CC:]]*X’, the optimisation can be applied
if the character class CC does not match character X. In the above
example, this holds because digit one is not in alpha character class.
This test is performed by mutually_exclusive_p function but it did not
check class bits of a charset opcode. This resulted in an assumption
that character classes do not match multibyte characters. For example,
it would incorrectly conclude that [[:alpha:]] doesn’t match ‘ż’.
This, in turn, led to the aforementioned Kleene star optimisation being
incorrectly applied in patterns such as ‘[[:graph:]]*☠’ (which should
match ‘☠’ but doesn’t as can be tested by executing
(string-match-p "[[:graph:]]*☠" "☠")
which should return 0 but instead yields nil.
This issue affects any class witch matches multibyte characters, i.e.
if ‘[[:cc:]]’ matches a multibyte character X then ‘[[:cc:]]*X’ will
fail to match ‘X’.
* src/regex.c (executing_charset): A new function for executing the
charset and charset_not opcodes. It performs check on the character
taking into consideration existing bitmap, range table and class bits.
It also advances the pointer in the regex bytecode past the parsed
opcode.
(CHARSET_LOOKUP_RANGE_TABLE_RAW, CHARSET_LOOKUP_RANGE_TABLE): Removed.
Code now included in executing_charset.
(mutually_exclusive_p, re_match_2_internal): Changed to take advantage
of executing_charset function.
* test/src/regex-tests.el: New file with tests for the character class
matching.
* src/emacs.c (main) [WINDOWSNT]: Move init_environment calls after the
set_initial_environment call. This prevents Emacs' modifications to the
environment from contaminating Vprocess_environment and
Vinitial_environment (Bug #10980).
* src/callproc.c (getenv_internal) [WINDOWSNT]: Consult Emacs' internal
environment in as a fallback to Vprocess_environment.
* test/src/callproc-tests.el (initial-environment-preserved): New Test.
Some tests are marked as expected to fail.
* test/lisp/international/ucs-normalize-tests.el: New tests.
* admin/unidata/NormalizationTest.txt: Add data for tests.
* admin/unidata/README: Add URL for NormalizationTest.txt.
* admin/notes/unicode: Add note about running (and updating the data
for) the new tests. Remove note about normalization being unsupported.
* dired.el (dired-always-read-filesystem): Add new option.
(dired-mark-files-containing-regexp): Use it (Bug#22694).
* doc/emacs/dired.texi: Mention it in the manual.
* test/lisp/dired-tests.el (dired-test-bug22694): Add test.
;* etc/NEWS: Add entry for this change.
* lisp/help-fns.el (describe-function-1): Check for aliases
defined with (defalias alias (symbol-function built-in)) (Bug#23887).
* test/lisp/help-fns-tests.el (help-fns-test-bug23887): Untag as
failing this test.
* lisp/net/tramp-gvfs.el (tramp-gvfs-get-remote-uid-gid-in-progress):
New variable.
(tramp-gvfs-maybe-open-connection): Use it.
* test/lisp/net/tramp-tests.el (tramp-test18-file-attributes):
Reorder test.
* src/editfns.c (styled_format): Don't include padding on the left
in the properties at the beginning of the string. (Bug#23897)
* test/src/editfns-tests.el (format-properties): Add tests for
faces when the string is padded on the left or on the right.
* src/textprop.c (extend_property_ranges): Accept an additional
argument OLD_END, and only extend the end of a property range if
its original end is at OLD_END; all the other ranges are left
intact. (Bug#23897)
* src/editfns.c (styled_format): Pass the original length of the
string to 'extend_property_ranges'.
* src/intervals.h (extend_property_ranges): Adjust prototype.
* test/src/editfns-tests.el (format-properties): Add tests for
bug#23897.
* lisp/textmodes/css-mode.el (css--complete-property-value): Make
"initial" and "unset" completion candidates for all CSS properties,
just like "inherit".
* test/lisp/textmodes/css-mode-tests.el
(css-test-complete-property-value): Update test to reflect the above
change.
* lisp/simple.el (delete-trailing-whitespace): Set newline’s character
syntax to non-whitespace so that ‘\s-’ regular expression does not match
it.
This simplifies the loop slightly since a simple ‘\s-+$’ can be used and
as a consequence ‘line-beginning-position’ function does not need to be
called any longer.
Furthermore, when newline has whitespace syntax, ‘\s-$’ regular
expression ends up matching empty lins since ‘\s-’ matches newline
characetr of proceeding line. This leads to needless loop iterations.
Since previous change to ‘delete-trailing-whitespace’ already introduced
‘with-syntax-table’, take advantage of it and also overwrite newline’s
character syntax.
* lisp/simple.el (delete-trailing-whitespace): Treat form fead as
a non-whitespace character (regradless of whether it’s character syntax
is whitespace) and delete any whitespace following it instead of leaving
lines with form feeds completely unchanged. I.e. a line like "\f " will
now became "\f".
* lisp/progmodes/ruby-mode.el (ruby-toggle-string-quotes):
Handle quoting of the first character in the string.
* test/lisp/progmodes/ruby-mode-tests.el
(ruby-toggle-string-quotes-quotes-correctly): Update.
* lisp/progmodes/ruby-mode.el (ruby-toggle-string-quotes):
Change logic to quote based on the current quote of the string.
* test/lisp/progmodes/ruby-mode-tests.el
(ruby-toggle-string-quotes-quotes-correctly): Add test.
Copyright-paperwork-exempt: yes
* lisp/net/tramp-sh.el (tramp-methods) <rsync>: Add "-p" and
"-s" arguments.
(tramp-do-copy-or-rename-file-out-of-band):
Call `file-name-directory' also for remote NEWNAME.
* test/lisp/net/tramp-tests.el (tramp-test15-copy-directory):
Do not skip for tramp-smb.el. Test als COPY-CONTENTS case.
(tramp-test24-file-name-completion): Improve check for
hostname completion.
(tramp--test-rsync-p): New defun.
(tramp-test31-special-characters)
(tramp-test31-special-characters-with-stat)
(tramp-test31-special-characters-with-perl)
(tramp-test31-special-characters-with-ls, tramp-test32-utf8)
(tramp-test32-utf8-with-stat, tramp-test32-utf8-with-perl)
(tramp-test32-utf8-with-ls): Skip for "rsync".
* lisp/net/tramp-smb.el (tramp-smb-handle-copy-directory):
Check also for CIFS capabilities.
* test/lisp/filenotify-tests.el (file-notify--test-with-events):
Fix thinko in test.
(file-notify-test02-events, file-notify-test04-file-validity)
(file-notify-test07-backup)
(file-notify-test08-watched-file-in-watched-dir): Make test
conditions more weak. (Bug#23618)
* src/chartab.c (char_table_set_range): Start the loop from the
first character of the block to which FROM belongs. (Bug#23797)
* test/src/chartab-tests.el: New test file.
Make sure seq-contains return the element of the sequence instead of t.
* lisp/emacs-lisp/seq.el (seq-contains): Fix the function.
* test/lisp/emacs-lisp/seq-tests.el: Add a regression test.
* lisp/progmodes/cc-mode.el (c-or-c++-mode): A new function which
analyses contents of the buffer to determine whether it looks like C++
source code and based on that enables c-mode or c++-mode.
(c-or-c++-mode--regexp): Regular expression which, when matches
a buffer, signals file is C++.
* lisp/emacs-lisp/ert-x.el (ert-with-function-mocked): Remove macro
in favour of ‘cl-letf’ macro which is more generic. All existing
uses are migrated accordingly. The macro has not been included in
an official release yet so it should be fine to delete it.
* test/lisp/net/network-stream-tests.el (connect-to-tls-ipv4-wait)
(connect-to-tls-ipv4-nowait, connect-to-tls-ipv6-nowait):
Ensure gnutls-serv process gets killed.
(echo-server-nowait, connect-to-tls-ipv4-nowait):
Limit the amount of time we might wait.
* test/lisp/emacs-lisp/package-resources/package-test-server.py:
Do not hard-code port.
* test/lisp/emacs-lisp/package-tests.el (package-test-update-archives-async):
Update for the above change.
* test/lisp/filenotify-tests.el (file-notify--test-timeout):
Reintroduce value for cygwin, it's needed on slow systems.
(file-notify--wait-for-events): Move up definition.
(file-notify--test-no-descriptors): Use `file-notify--wait-for-events'.
(file-notify--test-with-events-check, file-notify--test-with-events)
(file-notify-test08-watched-file-in-watched-dir):
Use :random rather than `random.
(file-notify-test06-many-events): Do not skip for cygwin.
* test/lisp/filenotify-tests.el (file-notify--test-timeout):
Remove special case for Cygwin.
(file-notify-test07-backup): Update expected results on Cygwin.
