From 9d6162053ebb508dee02baabfae26dea053446c4 Mon Sep 17 00:00:00 2001 From: Andrea Corallo Date: Tue, 2 Nov 2021 21:01:20 +0100 Subject: [PATCH 1/7] * src/pdumper.c (dump_do_dump_relocation): Add sanity check. --- src/pdumper.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/pdumper.c b/src/pdumper.c index 11c680d77b7..a8f8d6fa00b 100644 --- a/src/pdumper.c +++ b/src/pdumper.c @@ -5295,6 +5295,9 @@ dump_do_dump_relocation (const uintptr_t dump_base, error ("Trying to load incoherent dumped eln file %s", SSDATA (comp_u->file)); + if (!CONSP (comp_u->file)) + error ("Incoherent compilation unit for dump was dumped"); + /* emacs_execdir is always unibyte, but the file names in comp_u->file could be multibyte, so we need to encode them. */ From 6ab6b2f1c81148f4c77e2bac8d81ae7d6b6090c0 Mon Sep 17 00:00:00 2001 From: Phillip Lord Date: Wed, 3 Nov 2021 10:30:02 +0000 Subject: [PATCH 2/7] Clarify build directory structure * admin/nt/dist-build/README-scripts: Clarify build directory structure --- admin/nt/dist-build/README-scripts | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/admin/nt/dist-build/README-scripts b/admin/nt/dist-build/README-scripts index f27bcd3bd66..26c3d1df72c 100644 --- a/admin/nt/dist-build/README-scripts +++ b/admin/nt/dist-build/README-scripts @@ -15,12 +15,16 @@ The file system needs to be organized like so: ~/emacs-build/git -Contains a checkout of the Emacs git repository, organized according -to branches, with git worktree +Contains checkouts and worktrees of the Emacs git repository, +organized according to branches. + +~/emacs-build/git/master + +A checkout out of the master branch of the Emacs git repository. ~/emacs-build/git/emacs-$branch -A branch of the git repository containing the current release +A worktree of the git repository containing the current release branch. This has to be created by hand. ~/emacs-build/git/emacs-$version From a16e66c68138801ea6d0de49d6889db1ce17c6fb Mon Sep 17 00:00:00 2001 From: Phillip Lord Date: Wed, 3 Nov 2021 11:03:00 +0000 Subject: [PATCH 3/7] Clarify build environment * admin/nt/dist-build/README-scripts: --- admin/nt/dist-build/README-scripts | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/admin/nt/dist-build/README-scripts b/admin/nt/dist-build/README-scripts index 26c3d1df72c..6b1adbe03e1 100644 --- a/admin/nt/dist-build/README-scripts +++ b/admin/nt/dist-build/README-scripts @@ -3,6 +3,15 @@ Distribution Build Scripts for Windows The scripts are used to build the binary distribution zip files for windows. +Environment +----------- + +A full installation of msys2 is required along for the build. The +various dependencies of Emacs need to be installed also. These change +over time, but are listed in build-deps-zips.py. + + + File System Organization ------------------------ @@ -22,12 +31,12 @@ organized according to branches. A checkout out of the master branch of the Emacs git repository. -~/emacs-build/git/emacs-$branch +~/emacs-build/git/emacs-$major-version A worktree of the git repository containing the current release branch. This has to be created by hand. -~/emacs-build/git/emacs-$version +~/emacs-build/git/emacs-$release-version A branch of the git repository containing the last release. The build-zips.sh file will create this for you. @@ -67,8 +76,8 @@ uploaded. Build Process ------------- -For each major version ----------------------- + +### For each major version The dependencies files need to be created. This can be around the time of the pre-tests, then used for all releases of that version, to @@ -92,8 +101,7 @@ files will be created in ~/emacs-upload from where they can be signed and uploaded with `gnupload`. -For snapshots from Master -------------------------- +### For snapshots from Master Snapshots are generally created from master when there is a release branch on which a release has already been created. At this point, @@ -114,8 +122,7 @@ used. Now, run `build-zips.sh -s` to build a snapshot release. -For snapshots from a Release Branch ------------------------------------ +### For snapshots from a Release Branch Snapshots can be built from a release branch; this is really only useful before a pre-test has happened. @@ -127,8 +134,8 @@ version number must be added to the command line with `build-zips.sh the version (e.g emacs-27-2019-12-26.zip) rather than than the Emacs version (e.g emacs-27.0.50.zip). -For snapshots from another branch ---------------------------------- + +### For snapshots from another branch Snapshots can be build from any other branch. There is rarely a need to do this, except where some significant, wide-ranging feature is From 81915a95af3a52c45dd0a63487c9ba08a6df4c3f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mattias=20Engdeg=C3=A5rd?= Date: Wed, 3 Nov 2021 13:42:25 +0100 Subject: [PATCH 4/7] Add manual section about how to avoid regexp problems Help users affected by our NFA engine's stack overflows and occasional poor performance, replacing old text that was more limited in scope. * doc/lispref/elisp.texi (Top): * doc/lispref/searching.texi (Regular Expressions): Add menu entries. (Regexp Problems): New node. (Regexp Special): * etc/PROBLEMS: Remove superseded text. --- doc/lispref/elisp.texi | 1 + doc/lispref/searching.texi | 77 +++++++++++++++++++++++++++++++++----- etc/PROBLEMS | 12 ------ 3 files changed, 69 insertions(+), 21 deletions(-) diff --git a/doc/lispref/elisp.texi b/doc/lispref/elisp.texi index c4bd97bf815..6057691239f 100644 --- a/doc/lispref/elisp.texi +++ b/doc/lispref/elisp.texi @@ -1316,6 +1316,7 @@ Regular Expressions * Rx Notation:: An alternative, structured regexp notation. @end ifnottex * Regexp Functions:: Functions for operating on regular expressions. +* Regexp Problems:: Some problems and how they may be avoided. Syntax of Regular Expressions diff --git a/doc/lispref/searching.texi b/doc/lispref/searching.texi index d27cfb8c0c7..ce79765b733 100644 --- a/doc/lispref/searching.texi +++ b/doc/lispref/searching.texi @@ -263,6 +263,7 @@ case-sensitive. * Rx Notation:: An alternative, structured regexp notation. @end ifnottex * Regexp Functions:: Functions for operating on regular expressions. +* Regexp Problems:: Some problems and how they may be avoided. @end menu @node Syntax of Regexps @@ -343,15 +344,6 @@ first tries to match all three @samp{a}s; but the rest of the pattern is The next alternative is for @samp{a*} to match only two @samp{a}s. With this choice, the rest of the regexp matches successfully. -@strong{Warning:} Nested repetition operators can run for a very -long time, if they lead to ambiguous matching. For -example, trying to match the regular expression @samp{\(x+y*\)*a} -against the string @samp{xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxz} could -take hours before it ultimately fails. Emacs may try each way of -grouping the @samp{x}s before concluding that none of them can work. -In general, avoid expressions that can match the same string in -multiple ways. - @item @samp{+} @cindex @samp{+} in regexp is a postfix operator, similar to @samp{*} except that it must match @@ -1884,6 +1876,73 @@ variables that may be set to a pattern that actually matches something. @end defvar +@node Regexp Problems +@subsection Problems with Regular Expressions +@cindex regular expression problems +@cindex regexp stack overflow +@cindex stack overflow in regexp + +The Emacs regexp implementation, like many of its kind, is generally +robust but occasionally causes trouble in either of two ways: matching +may run out of internal stack space and signal an error, and it can +take a long time to complete. The advice below will make these +symptoms less likely and help alleviate problems that do arise. + +@itemize +@item +Anchor regexps at the beginning of a line, string or buffer using +zero-width assertions (@samp{^} and @code{\`}). This takes advantage +of fast paths in the implementation and can avoid futile matching +attempts. Other zero-width assertions may also bring benefits by +causing a match to fail early. + +@item +Avoid or-patterns in favour of character alternatives: write +@samp{[ab]} instead of @samp{a\|b}. Recall that @samp{\s-} and @samp{\sw} +are equivalent to @samp{[[:space:]]} and @samp{[[:word:]]}, respectively. + +@item +Since the last branch of an or-pattern does not add a backtrack point +on the stack, consider putting the most likely matched pattern last. +For example, @samp{^\(?:a\|.b\)*c} will run out of stack if trying to +match a very long string of @samp{a}s, but the equivalent +@samp{^\(?:.b\|a\)*c} will not. + +(It is a trade-off: successfully matched or-patterns run faster with +the most frequently matched pattern first.) + +@item +Try to ensure that any part of the text can only match in a single +way. For example, @samp{a*a*} will match the same set of strings as +@samp{a*}, but the former can do so in many ways and will therefore +cause slow backtracking if the match fails later on. Make or-pattern +branches mutually exclusive if possible, so that matching will not go +far into more than one branch before failing. + +Be especially careful with nested repetitions: they can easily result +in very slow matching in the presence of ambiguities. For example, +@samp{\(?:a*b*\)+c} will take a long time attempting to match even a +moderately long string of @samp{a}s before failing. The equivalent +@samp{\(?:a\|b\)*c} is much faster, and @samp{[ab]*c} better still. + +@item +Don't use capturing groups unless they are really needed; that is, use +@samp{\(?:@dots{}\)} instead of @samp{\(@dots{}\)} for bracketing +purposes. + +@ifnottex +@item +Consider using @code{rx} (@pxref{Rx Notation}); it can optimise some +or-patterns automatically and will never introduce capturing groups +unless explicitly requested. +@end ifnottex +@end itemize + +If you run into regexp stack overflow despite following the above +advice, don't be afraid of performing the matching in multiple +function calls, each using a simpler regexp where backtracking can +more easily be contained. + @node Regexp Search @section Regular Expression Searching @cindex regular expression searching diff --git a/etc/PROBLEMS b/etc/PROBLEMS index daff102a0d3..ede83a6e7c0 100644 --- a/etc/PROBLEMS +++ b/etc/PROBLEMS @@ -742,18 +742,6 @@ completed" message that tls.el relies upon, causing affected Emacs functions to hang. To work around the problem, use older or newer versions of gnutls-cli, or use Emacs's built-in gnutls support. -*** Stack overflow in regexp matcher. -Due to fundamental limitations in the way Emacs' regular expression -engine is designed, you might run into combinatorial explosions in -backtracking with certain regexps. - -Avoid "\(...\(...\)*...\)*" and "\(...\)*\(...\)*". Look for a way to -anchor your regular expression, to avoid matching the null string in -infinite ways. The latter is what creates backtrack points, and -eventual overflow in practice. - -(Also prefer "\(?:...\)" to "\(...\)" unless you need the latter.) - * Runtime problems related to font handling ** Characters are displayed as empty boxes or with wrong font under X. From 4df99985640b6add97c50165e9ebc122b3396158 Mon Sep 17 00:00:00 2001 From: Eli Zaretskii Date: Wed, 3 Nov 2021 14:57:51 +0200 Subject: [PATCH 5/7] Fix header-line and tab-line when mode-line-compact is set * src/xdisp.c (display_mode_line): Make 'mode-line-compact' apply only to mode lines. (Bug#51558) --- src/xdisp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/xdisp.c b/src/xdisp.c index 436153b2bae..c05e7edbc97 100644 --- a/src/xdisp.c +++ b/src/xdisp.c @@ -25592,7 +25592,8 @@ display_mode_line (struct window *w, enum face_id face_id, Lisp_Object format) push_kboard (FRAME_KBOARD (it.f)); record_unwind_save_match_data (); - if (NILP (Vmode_line_compact)) + if (NILP (Vmode_line_compact) + || face_id == HEADER_LINE_FACE_ID || face_id == TAB_LINE_FACE_ID) { mode_line_target = MODE_LINE_DISPLAY; display_mode_element (&it, 0, 0, 0, format, Qnil, false); From 35bc2117de96bed24652eecdcd9a0d791b1f2e6d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Mart=C3=ADn?= Date: Tue, 2 Nov 2021 19:45:18 +0100 Subject: [PATCH 6/7] Document a macOS error message when opening Emacs * etc/PROBLEMS: Document a potential error message when opening Emacs on macOS for the first time. --- etc/PROBLEMS | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/etc/PROBLEMS b/etc/PROBLEMS index ede83a6e7c0..f08ca45953b 100644 --- a/etc/PROBLEMS +++ b/etc/PROBLEMS @@ -2737,6 +2737,21 @@ If you do, please send it to bug-gnu-emacs@gnu.org so we can list it here. * Runtime problems specific to macOS +** Error "Emacs can't be opened because Apple cannot check it for malicious software". + +Apple may incorrectly catalog Emacs as potentially malicious software +and show an error that resembles the following when you try to open +Emacs for the first time: + + "Emacs" can't be opened because Apple cannot check it for malicious + software. This software needs to be updated. Contact the developer + for more information. + +To avoid this alert, open Finder, go to Applications, control-click +the Emacs app icon, and then choose Open. This adds a security +exception for Emacs and from now on you should be able to open it by +double-clicking on its icon, like any other app. + ** macOS doesn't come with libxpm, so only XPM3 is supported. Libxpm is available for macOS as part of the XQuartz project. From 3e4eb54db49e23b593664e4a3de3d3d744de61b7 Mon Sep 17 00:00:00 2001 From: Eli Zaretskii Date: Wed, 3 Nov 2021 20:23:58 +0200 Subject: [PATCH 7/7] ; * etc/PROBLEMS: Minor copyedits of the last change. (Bug#51571) --- etc/PROBLEMS | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/etc/PROBLEMS b/etc/PROBLEMS index f08ca45953b..8055d0e4e2b 100644 --- a/etc/PROBLEMS +++ b/etc/PROBLEMS @@ -2737,16 +2737,18 @@ If you do, please send it to bug-gnu-emacs@gnu.org so we can list it here. * Runtime problems specific to macOS -** Error "Emacs can't be opened because Apple cannot check it for malicious software". +** Error message when opening Emacs on macOS -Apple may incorrectly catalog Emacs as potentially malicious software -and show an error that resembles the following when you try to open -Emacs for the first time: +When opening Emacs, you may see an error message saying something like +this: "Emacs" can't be opened because Apple cannot check it for malicious software. This software needs to be updated. Contact the developer for more information. +The reason is that Apple incorrectly catalogs Emacs as potentially +malicious software and thus shows this error message. + To avoid this alert, open Finder, go to Applications, control-click the Emacs app icon, and then choose Open. This adds a security exception for Emacs and from now on you should be able to open it by