Expiring package metadata is done by checking the timestamp in package
archive file. This is intended to limit the effectiveness of a replay
attack. The onus is on the package archives to implement a secure and
reasonable policy. (Debian uses 7 days before metadata expires.)
Together with package checksums, this adds sufficient protection
against metadata replay attacks. (Bug#19479)
* lisp/emacs-lisp/package.el (package-check-timestamp): New defcustom.
(bad-timestamp): New error.
(package--parse-header-from-buffer)
(package--parse-valid-until-from-buffer)
(package--parse-last-updated-from-buffer)
(package--archive-verify-timestamp)
(package--archive-verify-not-expired)
(package--compare-archive-timestamps)
(package--check-archive-timestamp): New defuns.
(package--download-one-archive): Check timestamp of the
'archive-contents' file using above functions. It is only checked if
it exists, which makes this change backwards compatible.
* lisp/calendar/iso8601.el (iso8601-parse): Add autoload cookie.
* test/lisp/emacs-lisp/package-tests.el
(package-test-parse-valid-until-from-buffer)
(package-test-parse-last-updated-from-buffer)
(package-test-archive-verify-timestamp)
(package-test-check-archive-timestamp)
(package-test-check-archive-timestamp/not-expired)
(package-test-check-archive-timestamp/expired): New tests.
* test/lisp/emacs-lisp/package-resources/archives/older/archive-contents:
* test/lisp/emacs-lisp/package-resources/archives/newer/archive-contents:
New files.
* doc/lispref/package.texi (Package Archives, Archive Web Server):
Document how to increase the security of a package archive using
checksums, signing and timestamps.
Package checksum verification is the first step towards protecting
users of package.el against replay attacks. Signing individual
packages still allows a hostile actor to distribute an out-of-date
package containing a known security defect. To avoid that, we need to
distribute signed package metadata (the ELPA "archive-contents" file)
and checksums for the individual packages together. (Bug#19479)
A subsequent patch will add support for last-update and expiration
timestamps in "archive-contents", without which the protection against
replay attacks will be largely ineffective.
Taken together, this feature will make signing individual packages
obsolete. We will instead rely on signing the metadata, package
checksums and timestamps. Note that individual package signatures
should still be distributed for a long time still to support old
versions of Emacs.
For more on replay attacks, see:
https://www2.cs.arizona.edu/stork/packagemanagersecurity/attacks-on-package-managers.html
* lisp/emacs-lisp/package.el (package-verify-checksums): New
defcustom.
(package-desc, package--ac-desc)
(package--add-to-archive-contents, package-install-from-archive): New
fields 'size' and 'checksums'.
(package-desc-filename): New function.
(package-error): New error type.
(bad-signature): Inherit from error type 'package-error'.
(bad-checksum, bad-size): New error types.
(package-insecure-hash-algorithms): New constant.
(package--verify-package-checksum)
(package--verify-package-size): New function to verify that the
checksum and size of a package corresponds to the checksum and size
data in the "archive-contents" file on the package archive.
(package--show-verify-checksum-error): New function to show
details of an error on checksum verification.
* lisp/emacs-lisp/package-x.el (package-upload-buffer-internal):
Update to use above new fields 'size' and 'checksums'.
* test/lisp/emacs-lisp/package-tests.el (package-test-refresh-contents)
(package-test-install-single-from-archive)
(package-test-list-filter-by-archive)
(package-test-list-filter-by-status): Update tests.
(with-install-using-checksum): New macro.
(package-test-install-wrong-size-single)
(package-test-install-wrong-size-tar): New tests for size checking.
(package-test-install-with-checksum/single-valid)
(package-test-install-with-checksum/single-invalid)
(package-test-install-with-checksum/tar-valid)
(package-test-install-with-checksum/tar-invalid): New tests for
installing packages with checksums.
(package-test-verification-text)
(package-tests-valid-md5-checksum)
(package-tests-valid-sha256-checksum)
(package-tests-valid-sha512-checksum): New variables.
(package-tests--run-verify-checksums-test): New macro.
(package-test-verify-package-checksums-nil/ignore-invalid)
(package-test-verify-package-checksums-allow-missing)
(package-test-verify-package-checksums-allow-missing/missing)
(package-test-verify-package-checksums-allow-missing/ignore-unsupported)
(package-test-verify-package-checksums-t)
(package-test-verify-package-checksums-t/invalid-fails)
(package-test-verify-package-checksums-t/missing-fails)
(package-test-verify-package-checksums-all)
(package-test-verify-package-checksums-all/invalid-fails)
(package-test-verify-package-checksums-all/missing-fails)
(package-test-verify-package-checksums-all/no-supported-hash-fails)
(package-test-verify-package-checksums-all/ignore-unsupported)
(package-test-verify-package-size): New tests for the checksum
support.
* test/lisp/emacs-lisp/package-resources/archive-contents:
* test/lisp/emacs-lisp/package-resources/checksum-invalid-1.0.el:
* test/lisp/emacs-lisp/package-resources/checksum-valid-123.el:
* test/lisp/emacs-lisp/package-resources/checksum-valid-tar-0.99.tar:
* test/lisp/emacs-lisp/package-resources/checksum-valid-tar-0.99.tar:
New test data files.
* doc/emacs/package.texi (Package Installation): Document package
checksum checking.
* etc/NEWS: Announce it.
* test/lisp/cus-edit-tests.el (cus-edit-tests-customize-saved/show-obsolete):
Add a fake saved-value property, to be able check that the obsolete
option is present in the Customize buffer. Expect the test to pass
now.
Set the default value when `set` encounters a PER_BUFFER variable
which has been let-bound globally, to match the behavior seen with
`make-variable-buffer-local`.
* test/src/data-tests.el (binding-test--let-buffer-local):
Add corresponding test.
(data-tests--set-default-per-buffer): Add tentative test for the
performance problem encountered in bug#41029.
All-ASCII strings cannot have substrings with non-ASCII characters in
them; use this fact to avoid searching entirely.
* src/fns.c (Fstring_search): For multibyte non-ASCII needle and
unibyte haystack, don't check if the haystack is all-ASCII; it's a
waste of time. For multibyte non-ASCII needle and multibyte
all-ASCII haystack, fail immediately.
* test/src/fns-tests.el (string-search): Add more test cases.
Also add some tests for braces and parse-partial-sexp amongst Lisp style
comments.
* test/src/syntax-tests.el (\;-in, \;-out): Add syntax for { and }.
(top-level): Add new tests for Lisp style comments.
(\#|-in, \#|-out): New functions.
(top-level): Add new tests for nested Lisp style comments, and mixtures of
nested comments with "ordinary" comments.
* test/src/syntax-resources/syntax-comments.txt (top-level): Add new test
fragments for #|...|#, etc.
* lisp/progmodes/cperl-mode.el (cperl-find-pods-heres): Add
"return" to the keywords which start a regex.
* lisp/progmodes/perl-mode.el (defconst): Add "return" to
'perl--syntax-exp-intro-keywords' (Bug#26850).
* test/lisp/progmodes/cperl-mode-tests.el (cperl-test-bug-28650):
New test (bug#26850).
* lisp/help.el (describe-map): Fix running `menu-item' :filter
functions. This fixes a mistake in the previous conversion of this
defun from the old C function describe_map. See the discussion in
Bug#39149.
* test/src/keymap-tests.el
(keymap---get-keyelt/runs-menu-item-filter)
(describe-buffer-bindings/menu-item-filter-show-binding)
(describe-buffer-bindings/menu-item-filter-hide-binding):
New tests.
(keymap-tests--test-menu-item-filter): New defun.
* lisp/net/tramp.el (tramp-handle-write-region):
* lisp/net/tramp-adb.el (tramp-adb-handle-write-region):
* lisp/net/tramp-sh.el (tramp-sh-handle-write-region):
* lisp/net/tramp-smb.el (tramp-smb-handle-write-region): Use `current-time'
if needed.
* lisp/net/tramp-gvfs.el (tramp-gvfs-gio-mapping):
(tramp-gvfs-do-copy-or-rename-file): Remove "gvfs-rename", it is
not trustworthy.
* test/lisp/net/tramp-tests.el (tramp-test07-file-exists-p): Check also for
symlinked files in trash.
(tramp-test20-file-modes): Revert last change, it was a thinko.
* test/lisp/progmodes/cperl-mode-tests.el
(cperl--run-test-cases): New macro, factored out from various
indentation / rewriting tests. Contains documentation of the
format used by the cperl-mode-resources files.
(cperl-test-bug-19709): Replace 'next-line' by 'forward-line'.
(cperl-test-indent-exp),
(cperl-test-indent-styles),
(cperl-test-bug-30393): Use the new macro.
(cperl-test-bug-19709): Make fit for Emacs 26.
(cperl-test-indent-styles): Skip for Perl mode (bug#44561).
* test/lisp/emacs-lisp/rx-tests.el (rx-compat): Use with-no-warnings
instead of with-suppressed-warnings which complains when running
the test interactively.
This is a regression from Emacs 26.
Reported by Phillip Stephani.
* lisp/emacs-lisp/rx.el (rx--pcase-transform): Process ? and ?? correctly.
* test/lisp/emacs-lisp/rx-tests.el (rx-pcase): Add test case.
(cherry picked from commit 575b0681d9)
This is a regression from Emacs 26.
Reported by Phillip Stephani.
* lisp/emacs-lisp/rx.el (rx--pcase-transform): Process ? and ?? correctly.
* test/lisp/emacs-lisp/rx-tests.el (rx-pcase): Add test case.
* lisp/progmodes/cperl-mode.el (cperl-style-alist): Add
cperl-close-paren-offset to the settings for PBP style.
* test/lisp/progmodes/cperl-mode-tests.el (cperl-bug19709):
New test to verify correct indentation of closing parentheses (Bug#19709).
* test/lisp/progmodes/cperl-mode-resources/cperl-bug-19709.pl:
New test case with code from the bug report.
* test/lisp/progmodes/cperl-mode-resources/cperl-indent-styles.pl:
Add a new test clause for cperl-close-paren-offset.
* test/src/keymap-tests.el
(keymap-keymap-set-parent/returns-parent): Remove test for the return
value of set-keymap-parent. It is not clear that returning the value
is a very good idea.
Problem pointed out by Stefan Monnier <monnier@iro.umontreal.ca>.
* lisp/progmodes/compile.el (compilation-error-regexp-alist-alist):
'javac': Column numbers are 1-based by default; remove subtraction and
η-reduce. Translate regexp to rx (mechanised).
* test/lisp/progmodes/compile-tests.el (compile-tests--test-regexps-data):
Adapt 'javac' test cases to the change.
The variable now only controls whether characters are printed, not
the radix. Control chars are printed in human-readable syntax
only when special escapes such as ?\n are available. Spaces,
formatting and combining chars are excluded (bug#44155).
Done in collaboration with Juri Linkov.
* src/character.c (graphic_base_p):
* src/print.c (named_escape): New functions.
(print_object): Change semantics as described above.
(syms_of_print): Rename integer-output-format. Update doc string.
* doc/lispref/streams.texi (Output Variables):
* etc/NEWS:
* test/src/print-tests.el (print-integers-as-characters):
Rename and update according to new semantics. The test now passes.
This library provides a fundamental reworking of the search
functionality previously found in nnir.el. It uses class-based search
engines to interface with external searching facilities, and a parsed
search query syntax that can search multiple engines.
* lisp/gnus/gnus-search.el: New library containing search
functionality for Gnus.
* doc/misc/gnus.texi: Document.
* lisp/gnus/gnus-group.el (gnus-group-make-search-group,
gnus-group-read-ephemeral-search-group): Remove references to nnir,
change meaning of prefix argument, change values of nnselect-function
and nnselect-args.
* lisp/gnus/nnselect.el: Replace references to nnir
(nnselect-request-article): Use gnus-search functions, and search
criteria.
(nnselect-request-thread, nnselect-search-thread): Use gnus-search
thread search.
(gnus-summary-make-search-group): Switch to use gnus-search function
and arguments.
* test/lisp/gnus/search-tests.el: Tests for new functionality.
* lisp/progmodes/cperl-mode.el (cperl-mode): Add a fix
which is only required for Emacs versions older than 27.
* test/lisp/progmodes/cperl-mode-tests.el (cperl-bug30393):
Add a test to verify correct indentation (bug#30393).
* test/lisp/progmodes/cperl-mode-tests.el (cperl-bug37127):
Skip this test for older Emacsen. The bug has been fixed
in Emacs, but outside of CPerl mode, and therefore will not
be available for older versions via ELPA.
* doc/lispref/files.texi (Contents of Directories):
Fix description of directory-files, directory-empty-p and
directory-files-and-attributes.
* etc/NEWS: Fix entry for directory-files-and-attributes. Fix typos.
* lisp/dired.el (directory-empty-p): Move function from here ...
* lisp/files.el (directory-empty-p): ... to here.
* lisp/net/ange-ftp.el (ange-ftp-directory-files): Call `nreverse' later.
* lisp/net/tramp.el (tramp-handle-directory-files):
* lisp/net/tramp-adb.el
(tramp-adb-handle-directory-files-and-attributes): Do not call
`nreverse'.
* src/dired.c (Fdirectory_files)
(Fdirectory_files_and_attributes): Fix docstrings.
* test/src/dired-tests.el: Removed. Tests moved to
test/lisp/dired-tests.el.
* test/lisp/dired-tests.el (dired-test-bug27899): Tag it :unstable.
(dired-test-directory-files)
(dired-test-directory-files-and-attributes): New tests.
* lisp/emacs-lisp/easy-mmode.el (easy-mmode--arg-docstring):
Only document the values we want to support, not the ones we
actually support.
(define-minor-mode): Partially revert to previous behaviour.
This fixes a regression with regards to the old C version of
substitute-command-keys.
* lisp/help.el (substitute-command-keys): Don't bind standard-output.
See Bug#39149.
* test/lisp/help-tests.el
(help-tests--was-in-buffer): New variable.
(help-substitute-command-keys/menu-filter-in-correct-buffer): New
test.
* lisp/help.el (describe-map-tree): Insert header into the original
buffer, not in standard-output.
* test/src/keymap-tests.el
(describe-buffer-bindings/header-in-current-buffer)
(describe-buffer-bindings/returns-nil): New tests.
Ref: https://debbugs.gnu.org/39149#31
* lisp/emacs-lisp/easy-mmode.el (easy-mmode--arg-docstring):
Clarify when minor modes are switched on/off when called from lisp
(bug#44341).
(define-minor-mode): Make calls from Lisp switch the mode on/off
as documented.
