Support expiration of metadata by package archives

Expiring package metadata is done by checking the timestamp in package
archive file.  This is intended to limit the effectiveness of a replay
attack.  The onus is on the package archives to implement a secure and
reasonable policy.  (Debian uses 7 days before metadata expires.)

Together with package checksums, this adds sufficient protection
against metadata replay attacks.  (Bug#19479)

* lisp/emacs-lisp/package.el (package-check-timestamp): New defcustom.
(bad-timestamp): New error.
(package--parse-header-from-buffer)
(package--parse-valid-until-from-buffer)
(package--parse-last-updated-from-buffer)
(package--archive-verify-timestamp)
(package--archive-verify-not-expired)
(package--compare-archive-timestamps)
(package--check-archive-timestamp): New defuns.
(package--download-one-archive): Check timestamp of the
'archive-contents' file using above functions.  It is only checked if
it exists, which makes this change backwards compatible.

* lisp/calendar/iso8601.el (iso8601-parse): Add autoload cookie.

* test/lisp/emacs-lisp/package-tests.el
(package-test-parse-valid-until-from-buffer)
(package-test-parse-last-updated-from-buffer)
(package-test-archive-verify-timestamp)
(package-test-check-archive-timestamp)
(package-test-check-archive-timestamp/not-expired)
(package-test-check-archive-timestamp/expired): New tests.

* test/lisp/emacs-lisp/package-resources/archives/older/archive-contents:
* test/lisp/emacs-lisp/package-resources/archives/newer/archive-contents:
New files.

* doc/lispref/package.texi (Package Archives, Archive Web Server):
Document how to increase the security of a package archive using
checksums, signing and timestamps.

doc/emacs/package.texi

@@ -338,20 +338,41 @@ name of the package archive directory.  You can alter this list if you
 wish to use third party package archives---but do so at your own risk,
 and use only third parties that you think you can trust!
 
-@anchor{Package Signing}
+@anchor{Package Checksums}
 @cindex package security
+@cindex package checksums
+  To improve security, maintainers of package archive can add two
+important measures: checksums and signatures of metadata.  When used
+together, they can increase your trust that a downloaded package has
+not been tampered with, and is not out of date.  Valid checksums and
+signatures are not a cast-iron guarantee that a package is not
+malicious, so you should still exercise caution.  Only install
+packages from package archives that you trust.
+
+  When installing a package from an archive providing package
+checksums, the package system will automatically verify that they
+match the downloaded package.  By default, Emacs will refuse to
+install a package with an invalid checksum, but still allow installing
+a package if checksums are missing.  To disable installing packages
+from archives without checksums, you can set user the user option
+@code{package-verify-checksums} to @code{t}.  This improves security,
+but requires that all package archives you use distribute checksums.
+
+@anchor{Package Signing}
 @cindex package signing
   The maintainers of package archives can increase the trust that you
-can have in their packages by @dfn{signing} them.  They generate a
+can have in their packages and package listings by @dfn{signing} them.
-private/public pair of cryptographic keys, and use the private key to
+They generate a private/public pair of cryptographic keys, and use the
-create a @dfn{signature file} for each package.  With the public key, you
+private key to create a @dfn{signature file} for the package listing
-can use the signature files to verify the package creator and make sure
+itself or each individual package.  With the public key, you can use
-the package has not been tampered with.  Signature verification uses
+the signature files to verify the files have not been tampered with.
-@uref{https://www.gnupg.org/, the GnuPG package} via the EasyPG
+Signature verification uses @uref{https://www.gnupg.org/, the GnuPG
-interface (@pxref{Top,, EasyPG, epa, Emacs EasyPG Assistant Manual}).
+package} via the EasyPG interface (@pxref{Top,, EasyPG, epa, Emacs
-A valid signature is not a cast-iron
+EasyPG Assistant Manual}).
-guarantee that a package is not malicious, so you should still
+
-exercise caution.  Package archives should provide instructions
+The public key for the GNU package archive is distributed with Emacs,
+in the @file{etc/package-keyring.gpg}.  Emacs uses it automatically.
+Other package archives should provide instructions
 on how you can obtain their public key.  One way is to download the
 key from a server such as @url{https://pgp.mit.edu/}.
 Use @kbd{M-x package-import-keyring} to import the key into Emacs.
@@ -361,8 +382,6 @@ subdirectory of @code{package-user-dir}, which causes Emacs to invoke
 GnuPG with the option @samp{--homedir} when verifying signatures.
 If @code{package-gnupghome-dir} is @code{nil}, GnuPG's option
 @samp{--homedir} is omitted.
-The public key for the GNU package archive is distributed with Emacs,
-in the @file{etc/package-keyring.gpg}.  Emacs uses it automatically.
 
 @vindex package-check-signature
 @vindex package-unsigned-archives

doc/lispref/package.texi

@@ -332,10 +332,22 @@ installing user.  (This is true for Emacs code in general, not just
 for packages.)  So you should ensure that your archive is
 well-maintained and keep the hosting system secure.
 
-  One way to increase the security of your packages is to @dfn{sign}
+  To increase the security of your packages, you should distribute
-them using a cryptographic key.  If you have generated a
+package checksums in the package metadata file
-private/public gpg key pair, you can use gpg to sign the package like
+@file{archive-contents}.  You should also @dfn{sign} the package
-this:
+metadata file using a cryptographic key.  Finally, it is important to
+include creation and expiration timestamps information in that file.
+
+  Signing individual packages is also supported, but considered
+obsolete.  It provides less security than package checksums, signing
+the @file{archive-contents} file, and creation and expiration
+timestamps does when used together.  More specifically, signing
+individual packages does not protect against ``replay attacks''.  Note
+that distributing signatures for individual packages is still
+recommended to support Emacs versions older than 28.1.
+
+  If you have generated a private/public gpg key pair, you can use gpg
+to sign a package or the @file{archive-contents} file like this:
 
 @c FIXME EasyPG / package-x way to do this.
 @example
@@ -371,6 +383,9 @@ Return a lisp form describing the archive contents. The form is a list
 of 'package-desc' structures (see @file{package.el}), except the first
 element of the list is the archive version.
 
+@item archive-contents.sig
+Return the signature for @file{archive-contents}.
+
 @item <package name>-readme.txt
 Return the long description of the package.
 

etc/NEWS

@@ -867,6 +867,26 @@ See the new user options 'package-name-column-width',
 'package-version-column-width', 'package-status-column-width', and
 'package-archive-column-width'.
 
+*** Support for package checksums.
+This improves the security of the Emacs package system.  If the
+package archives you use implements package checksums, you will
+automatically benefit from this by default.
+
+The user option 'package-verify-checksums' controls how and when the
+package system will use checksums.  The default is 'allow-missing',
+which will check them when they are available yet allow installation
+if they are missing.
+
+For improved security, you might want to set this to 't' or
+'all'.  Make sure that the package archives you use support checksums
+before setting these values, or you will be unable to install
+packages.
+
+*** Support expiration of package archive metadata.
+When a package archive distributes a last-updated and expiration
+timestamp, they will automatically be used to verify that distributed
+packages are not out of date.
+
 ** gdb-mi
 
 +++

lisp/calendar/iso8601.el

@@ -114,6 +114,7 @@
          iso8601--duration-week-match
          iso8601--duration-combined-match)))
 
+;;;###autoload
 (defun iso8601-parse (string &optional form)
   "Parse an ISO 8601 date/time string and return a `decode-time' structure.
 

lisp/emacs-lisp/package-x.el

@@ -219,7 +219,9 @@ if it exists."
 	  (let ((contents (or (package--archive-contents-from-url archive-url)
 			      (package--archive-contents-from-file)))
 		(new-desc (package-make-ac-desc
-                           split-version requires desc file-type extras)))
+                           split-version requires desc file-type extras
+                           ;; FIXME: Use better values than nil nil.
+                           nil nil)))
 	    (if (> (car contents) package-archive-version)
 		(error "Unrecognized archive version %d" (car contents)))
 	    (let ((elt (assq pkg-name (cdr contents))))

lisp/emacs-lisp/package.el

@@ -335,6 +335,40 @@ default directory."
   :risky t
   :version "26.1")
 
+(defcustom package-verify-checksums 'allow-missing
+  "Non-nil means to verify the checksum of a package before installing it.
+
+This can be one of:
+- t                Require a valid checksum; refuse to install
+                   package if the checksum is missing or invalid.
+                   Verify only one checksum.
+- `all'            Same as t, but verify all available (and supported)
+                   checksums.
+- `allow-missing'  Same as t if a checksum exists, but install a
+                   package even if there is no checksum.
+- nil              Ignore checksums.
+
+The package checksums are automatically fetched from package
+archives with the package data on `package-refresh-contents'.
+
+Note that setting this to nil is intended for debugging, and
+should normally not be used since it will decrease security."
+  :type '(choice (const nil :tag "Never")
+                 (const allow-missing :tag "Allow missing")
+                 (const t :tag "Require valid checksum")
+                 (const t :tag "Require valid checksum, and check all"))
+  :risky t
+  :version "28.1")
+
+(defcustom package-check-timestamp t
+  "Non-nil means to verify the package archive timestamp.
+
+Note that setting this to nil is intended for debugging, and
+should normally not be used since it will decrease security."
+  :type 'boolean
+  :risky t
+  :version "28.1")
+
 (defcustom package-check-signature 'allow-unsigned
   "Non-nil means to check package signatures when installing.
 More specifically the value can be:
@@ -417,6 +451,15 @@ synchronously."
   :type 'number
   :version "28.1")
 
+
+;;; Errors
+
+(define-error 'package-error "Unknown package error")
+(define-error 'bad-size "Package size mismatch" 'package-error)
+(define-error 'bad-signature "Failed to verify signature" 'package-error)
+(define-error 'bad-checksum "Failed to verify checksum" 'package-error)
+(define-error 'bad-timestamp "Failed to verify timestamp" 'package-error)
+
 
 ;;; `package-desc' object definition
 ;; This is the struct used internally to represent packages.
@@ -449,6 +492,8 @@ synchronously."
                                  requirements)))
                  (kind (plist-get rest-plist :kind))
                  (archive (plist-get rest-plist :archive))
+                 (checksums (plist-get rest-plist :checksums))
+                 (size (plist-get rest-plist :size))
                  (extras (let (alist)
                            (while rest-plist
                              (unless (memq (car rest-plist) '(:kind :archive))
@@ -486,6 +531,13 @@ Slots:
 
 `extras' Optional alist of additional keyword-value pairs.
 
+`size'  Size of the package in bytes.
+
+`checksums' Checksums for the package file.  Alist of ((ALGORITHM
+        . CHECKSUM)) where ALGORITHM is a symbol specifying a
+        `secure-hash' algorithm, and CHECKSUM is a string
+        containing the checksum.
+
 `signed' Flag to indicate that the package is signed by provider."
   name
   version
@@ -495,7 +547,9 @@ Slots:
   archive
   dir
   extras
-  signed)
+  signed
+  size
+  checksums)
 
 (defun package--from-builtin (bi-desc)
   "Create a `package-desc' object from BI-DESC.
@@ -558,6 +612,13 @@ Signal an error if the kind is none of the above."
     ('dir "")
     (kind (error "Unknown package kind: %s" kind))))
 
+(defun package-desc-filename (pkg-desc)
+  "Return file-name of package-desc object PKG-DESC.
+This is the concatenation of `package-desc-full-name' and
+`package-desc-suffix'."
+  (concat (package-desc-full-name pkg-desc)
+          (package-desc-suffix pkg-desc)))
+
 (defun package-desc--keywords (pkg-desc)
   "Return keywords of package-desc object PKG-DESC.
 These keywords come from the foo-pkg.el file, and in general
@@ -1334,7 +1395,88 @@ errors signaled by ERROR-FORM or by BODY).
                    url))
           (insert-file-contents-literally url)))))
 
-(define-error 'bad-signature "Failed to verify signature")
+(defun package--show-verify-checksum-error (pkg-desc details)
+  "Show error on failed checksum verification of PKG-DESC with DETAILS.
+Error is displayed in a new buffer named \"*Error*\"."
+  (with-output-to-temp-buffer "*Error*"
+    (with-current-buffer standard-output
+      (insert (format "Failed to verify checksum of package `%s':\n\n"
+                      (package-desc-name pkg-desc)))
+      (insert details))))
+
+(defconst package-insecure-hash-algorithms '(md5 sha1)
+  "List of hash algorithms that are not considered secure.")
+
+(defun package--verify-package-checksum (pkg-desc)
+  "Verify checksums of `package-desc' object PKG-DESC.
+This assumes that the we are in a buffer containing package.
+
+The value of `package-verify-checksums' decides what this
+function does:
+- t                Verify that there is at least one valid checksum.
+- `all'            Like t, but check all supported checksums.
+- `allow-missing'  Verify checksum if it exists, otherwise do
+                   nothing.
+- nil              Do nothing.
+
+Signal an error of type `bad-checksum' if the verification."
+  (cl-flet*
+      ((supported-hashes
+        (lambda ()
+          (or (seq-filter
+               (lambda (h)
+                 (and (memql (car h) (secure-hash-algorithms))
+                      (not (memql (car h) package-insecure-hash-algorithms))))
+               (package-desc-checksums pkg-desc))
+              ;; Failed; signal error.
+              (package--show-verify-checksum-error
+               pkg-desc
+               (concat
+                (if (package-desc-checksums pkg-desc)
+                    (concat
+                     "No supported checksums found\n\n"
+                     (format-message "Package archive had: %s\n"
+                                     (package-desc-checksums pkg-desc))
+                     (format-message "Emacs supports: %s\n"
+                                     (secure-hash-algorithms)))
+                  "Package archive had no checksums for this package\n")))
+              (signal 'bad-checksum "no supported checksums found"))))
+       (do-check
+        (lambda (&optional all)
+          (dolist (hash (seq-take (supported-hashes)
+                                  (if all most-positive-fixnum 1)))
+            (let* ((algorithm (car hash))
+                   (expected (cdr hash))
+                   (actual (secure-hash algorithm (current-buffer))))
+              (if (equal expected actual) t
+                ;; Failed; signal error.
+                (package--show-verify-checksum-error
+                 pkg-desc
+                 (concat
+                  (format-message "\nChecksum mismatch (%s)\n\n" algorithm)
+                  (format-message "Expected: %s\n" expected)
+                  (format-message "Result: %s\n" actual)))
+                (signal 'bad-checksum (list "checksum mismatch" expected actual))))))))
+    (pcase package-verify-checksums
+      ('nil nil)
+      ('allow-missing (when (package-desc-checksums pkg-desc) (do-check)))
+      ('t (do-check))
+      ('all (do-check 'all))
+      (_ (user-error "Value of `package-verify-checksums' is invalid: `%s'"
+                     package-verify-checksums)))))
+
+(defun package--verify-package-size (pkg-desc)
+  "Verify package size of `package-desc' object PKG-DESC.
+This assumes that the we are in a buffer containing package."
+  (when-let ((expected (package-desc-size pkg-desc))
+             (actual (string-bytes (buffer-string))))
+    (unless (equal expected actual)
+      (with-output-to-temp-buffer "*Error*"
+        (with-current-buffer standard-output
+          (insert (format "Mismatch in package size for `%s':\n"
+                          (package-desc-name pkg-desc)))
+          (insert (format "Expected %s bytes, but received %s" expected actual))))
+      (signal 'bad-size (list "size mismatch" expected actual)))))
 
 (defun package--check-signature-content (content string &optional sig-file)
   "Check signature CONTENT against STRING.
@@ -1461,14 +1603,19 @@ the table."
                 (version-list-< table-version version))
         (puthash name version package--compatibility-table)))))
 
-;; Package descriptor objects used inside the "archive-contents" file.
-;; Changing this defstruct implies changing the format of the
-;; "archive-contents" files.
 (cl-defstruct (package--ac-desc
-               (:constructor package-make-ac-desc (version reqs summary kind extras))
+               (:constructor
+                package-make-ac-desc (version reqs summary kind extras size checksums))
                (:copier nil)
                (:type vector))
-  version reqs summary kind extras)
+  "Package descriptor object used inside the \"archive-contents\" file.
+Changing this defstruct implies changing the format of the
+\"archive-contents\" files.
+
+This is mainly used in `package--add-to-archive-contents' to make
+the code that parses the \"archive-contents\" file more
+readable."
+  version reqs summary kind extras size checksums)
 
 (defun package--append-to-alist (pkg-desc alist)
   "Append an entry for PKG-DESC to the start of ALIST and return it.
@@ -1506,10 +1653,14 @@ Also, add the originating archive to the `package-desc' structure."
            :summary (package--ac-desc-summary (cdr package))
            :kind (package--ac-desc-kind (cdr package))
            :archive archive
+           ;; Older "archive-contents" files might not have the
+           ;; below elements.
            :extras (and (> (length (cdr package)) 4)
-                        ;; Older archive-contents files have only 4
+                        (package--ac-desc-extras (cdr package)))
-                        ;; elements here.
+           :size (and (> (length (cdr package)) 5)
-                        (package--ac-desc-extras (cdr package)))))
+                      (package--ac-desc-size (cdr package)))
+           :checksums (and (> (length (cdr package)) 6)
+                           (package--ac-desc-checksums (cdr package)))))
          (pinned-to-archive (assoc name package-pinned-packages)))
     ;; Skip entirely if pinned to another archive.
     (when (not (and pinned-to-archive
@@ -1671,6 +1822,100 @@ Once it's empty, run `package--post-download-archives-hook'."
     (message "Package refresh done")
     (run-hooks 'package--post-download-archives-hook)))
 
+(defun package--parse-header-from-buffer (header name)
+  "Find and return \"archive-contents\" HEADER for archive NAME.
+This function assumes that the current buffer contains the
+\"archive-contents\" file.
+
+A valid header looks like: \";; HEADER: <TIMESTAMP>\"
+
+Where <TIMESTAMP> is a valid ISO-8601 (RFC 3339) date.  If there
+is such a line but <TIMESTAMP> is invalid, show a warning and
+return nil.  If there is no valid header, return nil."
+  (save-excursion
+    (goto-char (point-min))
+    (when (re-search-forward (concat "^;; " header ": *\\(.+?\\) *$") nil t)
+      (condition-case-unless-debug nil
+          (encode-time (iso8601-parse (match-string 1)))
+        (lwarn '(package timestamp)
+               (list (format "Malformed timestamp for archive `%s': `%s'"
+                             name (match-string 1))))))))
+
+(defun package--parse-valid-until-from-buffer (name)
+  "Find and return \"Valid-Until\" header for archive NAME."
+  (package--parse-header-from-buffer "Valid-Until" name))
+
+(defun package--parse-last-updated-from-buffer (name)
+  "Find and return \"Last-Updated\" header for archive NAME."
+  (package--parse-header-from-buffer "Last-Updated" name))
+
+(defun package--archive-verify-timestamp (new old name)
+  "Return t if timestamp NEW is more recent than OLD for archive NAME.
+Signal error otherwise.
+Warn if NEW is in the future."
+  ;; If timestamp is missing on cached (old) file, do nothing here.
+  ;; This package archive recently introduced support for timestamps.
+  ;; We will require a timestamp for that archive in future updates.
+  (if old
+      (cond
+       ((not new)
+        (signal 'bad-timestamp
+                (list (format-message
+                       (concat
+                        "New archive contents for `%s' missing "
+                        "timestamp, refusing to proceed")
+                       name))))
+       ((time-less-p new old)
+        (signal 'bad-timestamp
+                (list (format-message
+                       (concat
+                        "New archive contents for `%s' older than "
+                        "cached, refusing to proceed")
+                       name))))
+       ((time-less-p (current-time) new)
+        (signal 'bad-timestamp
+                (list (format-message
+                       (concat
+                        "New archive contents for `%s' is "
+                        "in the future: %s")
+                       name (format-time-string "%c" new)))))
+       ;; Check ok, return t.
+       (t))
+    t))
+
+(defun package--archive-verify-not-expired (timestamp name)
+  "Return t if TIMESTAMP has not yet expired for archive NAME.
+Signal error otherwise."
+  (unless (time-less-p (current-time) timestamp)
+    (signal 'bad-timestamp
+            (list (format-message
+                   (concat
+                    "Package archive `%s' has sent "
+                    "an expired `archive-contents' file")
+                   name)))))
+
+(defun package--check-archive-timestamp (name)
+  "Verify timestamp of \"archive-contents\" file for archive NAME.
+Compare the archive timestamp of the previously downloaded
+\"archive-contents\" file to the timestamp in the current buffer.
+Signal error if the old timestamp is more recent than the new one.
+
+Do nothing if there is no previously downloaded file, if such a
+file exists but does not contain any timestamp, or if
+`package-check-timestamp' is nil."
+  (let ((old-file (expand-file-name
+                   (concat "archives/" name "/archive-contents")
+                   package-user-dir)))
+    (when (and package-check-timestamp
+               (file-readable-p old-file))
+      (let ((old (with-temp-buffer
+                   (insert-file-contents old-file)
+                   (package--parse-last-updated-from-buffer name)))
+            (new (package--parse-last-updated-from-buffer name))
+            (new-expires (package--parse-valid-until-from-buffer name)))
+        (package--archive-verify-timestamp new old name)
+        (package--archive-verify-not-expired new-expires name)))))
+
 (defun package--download-one-archive (archive file &optional async)
   "Retrieve an archive file FILE from ARCHIVE, and cache it.
 ARCHIVE should be a cons cell of the form (NAME . LOCATION),
@@ -1684,6 +1929,7 @@ similar to an entry in `package-alist'.  Save the cached copy to
            (content (buffer-string))
            (dir (expand-file-name (concat "archives/" name) package-user-dir))
            (local-file (expand-file-name file dir)))
+      (package--check-archive-timestamp name)
       (when (listp (read content))
         (make-directory dir t)
         (if (or (not (package-check-signature))
@@ -1979,9 +2225,10 @@ if all the in-between dependencies are also in PACKAGE-LIST."
   (when (eq (package-desc-kind pkg-desc) 'dir)
     (error "Can't install directory package from archive"))
   (let* ((location (package-archive-base pkg-desc))
-         (file (concat (package-desc-full-name pkg-desc)
+         (file (package-desc-filename pkg-desc)))
-                       (package-desc-suffix pkg-desc))))
     (package--with-response-buffer location :file file
+      (package--verify-package-size pkg-desc)
+      (package--verify-package-checksum pkg-desc)
       (if (or (not (package-check-signature))
               (member (package-desc-archive pkg-desc)
                       package-unsigned-archives))

test/lisp/emacs-lisp/package-resources/archive-contents

@@ -14,4 +14,38 @@
  (multi-file .
              [(0 2 3)
               nil "Example of a multi-file tar package" tar
-              ((:url . "http://puddles.li"))]))
+              ((:url . "http://puddles.li"))])
+ (checksum-valid  .
+                  [(123)
+                    nil "A single-file package with a valid checksum." single
+                    nil
+                    343
+                    ((sha512 . "a889917427569cc6817db5db08a88390d44ec010acdf6810c2dfaba04b9a03f00315378c3f03d5f4d531833028ad61db54c4c56106662585da6a0dde602f5c0d"))])
+ (checksum-valid-tar .
+                     [(0 99)
+                     nil "A multi-file package with a valid checksum." tar
+                     nil
+                     10240
+                     ((sha512 . "2be7c37a16db32a2b08fc917ed5f4241814e2665bda1bd15328c2e5a842e45b81f6f31274697248ffaabf8010796685acb3342c5920af53ddd1e75d7fd764bd1"))])
+ (checksum-invalid .
+                   [(1 0)
+                    nil "A single-file package with an invalid checksum." single
+                    nil
+                    365
+                    ((sha512 . "not-a-valid-checksum"))])
+ (checksum-invalid-tar .
+                       [(0 1)
+                        nil "A multi-file package with an invalid checksum." tar
+                        nil
+                        10240
+                        ((sha512 . "not-a-valid-checksum"))])
+ (wrong-size-single .
+                    [(1 0)
+                     nil "A single-file package with an invalid size." single
+                     nil
+                     1])
+ (wrong-size-tar .
+                 [(1 0)
+                  nil "A multi-file package with an invalid size." tar
+                  nil
+                  1]))

test/lisp/emacs-lisp/package-resources/archives/newer/archive-contents

@@ -0,0 +1 @@
+;; Last-Updated: 2020-06-01T00:00:00.000Z

test/lisp/emacs-lisp/package-resources/archives/older/archive-contents

@@ -0,0 +1 @@
+;; Last-Updated: 2019-01-01T00:00:00.000Z

test/lisp/emacs-lisp/package-resources/checksum-invalid-1.0.el

@@ -0,0 +1,17 @@
+;;; checksum-invalid.el --- A package with an invalid checksum in archive-contents
+
+;; Version: 1.0
+
+;;; Commentary:
+
+;; This package has an invalid checksum in archive-contents and is
+;; just used to verify that package.el refuses to install.
+
+;;; Code:
+
+(defun p-equal-to-np-p ()
+  (error "FIXME"))
+
+(provide 'checksum-invalid)
+
+;;; checksum-invalid.el ends here

test/lisp/emacs-lisp/package-resources/checksum-valid-123.el

@@ -0,0 +1,17 @@
+;;; checksum-valid.el --- A package with an valid checksum in archive-contents
+
+;; Version: 123
+
+;;; Commentary:
+
+;; This package has an valid checksum in archive-contents and is
+;; used to verify that package.el installs it.
+
+;;; Code:
+
+(defun p-equal-to-np-p ()
+  (error "FIXME"))
+
+(provide 'checksum-valid)
+
+;;; checksum-valid.el ends here

test/lisp/emacs-lisp/package-resources/wrong-size-single-1.0.el

@@ -0,0 +1 @@
+;; This file just has the wrong size (i.e. not 1 as specified).

test/lisp/emacs-lisp/package-tests.el

@@ -44,6 +44,9 @@
 
 (setq package-menu-async nil)
 
+;; Silence byte-compiler.
+(defvar epg-config--program-alist)
+
 (defvar package-test-user-dir nil
   "Directory to use for installing packages during testing.")
 
@@ -304,14 +307,33 @@ Must called from within a `tar-mode' buffer."
   (with-package-test ()
     (package-initialize)
     (package-refresh-contents)
-    (should (eq 4 (length package-archive-contents)))))
+    (should (eq 10 (length package-archive-contents)))))
 
 (ert-deftest package-test-install-single-from-archive ()
   "Install a single package from a package archive."
   (with-package-test ()
     (package-initialize)
     (package-refresh-contents)
-    (package-install 'simple-single)))
+    (package-install 'simple-single)
+    (should (package-installed-p 'simple-single))))
+
+(ert-deftest package-test-install-wrong-size-single ()
+  "Install a tar package with invalid size."
+  (should-error
+   (with-package-test ()
+     (package-initialize)
+     (package-refresh-contents)
+     (package-install 'wrong-size-single))
+   :type 'bad-size))
+
+(ert-deftest package-test-install-wrong-size-tar ()
+  "Install a tar package with invalid size."
+  (should-error
+   (with-package-test ()
+     (package-initialize)
+     (package-refresh-contents)
+     (package-install 'wrong-size-tar))
+   :type 'bad-size))
 
 (ert-deftest package-test-install-prioritized ()
   "Install a lower version from a higher-prioritized archive."
@@ -389,8 +411,8 @@ Must called from within a `tar-mode' buffer."
     ;;       the testing environment currently only has one.
     (package-menu-filter-by-archive "gnu")
     (goto-char (point-min))
-    (should (looking-at "^\\s-+multi-file"))
+    (should (looking-at "^\\s-+checksum-invalid"))
-    (should (= (count-lines (point-min) (point-max)) 4))
+    (should (= (count-lines (point-min) (point-max)) 10))
     (should-error (package-menu-filter-by-archive "non-existent archive"))))
 
 (ert-deftest package-test-list-filter-by-keyword ()
@@ -416,7 +438,7 @@ Must called from within a `tar-mode' buffer."
     (package-menu-filter-by-status "available")
     (goto-char (point-min))
     (should (re-search-forward "^\\s-+multi-file" nil t))
-    (should (= (count-lines (point-min) (point-max)) 4))
+    (should (= (count-lines (point-min) (point-max)) 10))
     ;; No installed packages in default environment.
     (should-error (package-menu-filter-by-status "installed"))))
 
@@ -671,6 +693,230 @@ Must called from within a `tar-mode' buffer."
 		"Status: Installed in ['`‘]signed-good-1.0/['’]."
 		nil t))))))
 
+
+;;; Tests for package checksum verification.
+
+(defmacro with-install-using-checksum (ok fail package)
+  "Test installing PACKAGE while setting `package-verify-checksums'."
+  (declare (indent 2))
+  `(progn
+     (dolist (opt ,ok)
+       (let ((package-verify-checksums opt))
+         (with-package-test ()
+           (package-initialize)
+           (package-refresh-contents)
+           (package-install ,package)
+           (package-installed-p ,package))))
+     (dolist (opt ,fail)
+       (let ((package-verify-checksums opt))
+         (should-error
+          (with-package-test ()
+            (package-initialize)
+            (package-refresh-contents)
+            (package-install ,package))
+          :type 'bad-checksum)))))
+
+(ert-deftest package-test-install-with-checksum/single-valid ()
+  "Install a single package with valid checksum."
+  (with-install-using-checksum '(nil allow-missing t all) '() 'checksum-valid))
+
+(ert-deftest package-test-install-with-checksum/single-invalid ()
+  "Install a tar package with invalid checksum."
+  (with-install-using-checksum '(nil) '(allow-missing t all) 'checksum-invalid))
+
+(ert-deftest package-test-install-with-checksum/tar-valid ()
+  "Install a tar package with valid checksum."
+  (with-install-using-checksum '(nil allow-missing t all) '() 'checksum-valid-tar))
+
+(ert-deftest package-test-install-with-checksum/tar-invalid ()
+  "Install a tar package with invalid checksum."
+  (with-install-using-checksum '(nil) '(allow-missing t all) 'checksum-invalid-tar))
+
+(defconst package-test-verification-text
+  "Example text for testing checksum verification.")
+(defconst package-tests-valid-md5-checksum
+  ;; (secure-hash 'md5 package-test-verification-text)
+  "abe6375809e532f081b808b3aa052dfb")
+(defconst package-tests-valid-sha256-checksum
+  ;; (secure-hash 'sha256 package-test-verification-text)
+  "6875aa4523e45ddef627b4edf1296f1d7dd0c22ddd6a6584f0228215d25eefcd")
+(defconst package-tests-valid-sha512-checksum
+  ;; (secure-hash 'sha512 package-test-verification-text)
+  (concat "bdc631f9e675b1ea34570f0a4bb44568dc5cecac905eea737f5f451bc52fd0c6"
+          "81b0d8b3dc2a942b9950fbe9096ebdf517668245c9b5a7bbdea8487a8f9cdce6"))
+
+(defmacro package-tests--run-verify-checksums-test (verify-checksums checksums)
+  "Run a test for `package-verify-checksums'."
+  (declare (indent 1))
+  `(with-temp-buffer
+     (insert package-test-verification-text)
+     (let ((package-verify-checksums ,verify-checksums)
+           (pkg (package-desc-create :name 'foobar
+                              :version '(1 0)
+                              :summary "Just a package with checksum."
+                              :kind 'single
+                              :checksums ,checksums)))
+       (package--verify-package-checksum pkg))))
+
+(ert-deftest package-test-verify-package-checksums-nil/ignore-invalid ()
+  "Ignore all checksums even when invalid."
+  (package-tests--run-verify-checksums-test nil
+    '((sha512 . "invalid")
+      (invalid . "invalid"))))
+
+(ert-deftest package-test-verify-package-checksums-nil/ignore-empty ()
+  "Ignore all checksums even when empty."
+  (package-tests--run-verify-checksums-test nil
+    nil))
+
+(ert-deftest package-test-verify-package-checksums-allow-missing ()
+  "Verify checksums (allow-missing) -- verify if available."
+  (package-tests--run-verify-checksums-test 'allow-missing
+    `((sha512 . ,package-tests-valid-sha512-checksum))))
+
+(ert-deftest package-test-verify-package-checksums-allow-missing/missing ()
+  "Verify checksums (allow-missing) -- allow missing."
+  (package-tests--run-verify-checksums-test 'allow-missing
+    nil))
+
+(ert-deftest package-test-verify-package-checksums-allow-missing/ignore-unsupported ()
+  "Verify checksums (t) -- ignore unsupported algorithm."
+  (package-tests--run-verify-checksums-test 'allow-missing
+    `((ignore . "not supported")
+      (sha512 . ,package-tests-valid-sha512-checksum))))
+
+(ert-deftest package-test-verify-package-checksums-t ()
+  "Verify checksums (t) -- succeed when valid."
+  (package-tests--run-verify-checksums-test t
+    `((sha512 . ,package-tests-valid-sha512-checksum))))
+
+(ert-deftest package-test-verify-package-checksums-t/invalid-fails ()
+  "Verify checksums (t) -- fail on invalid."
+  (should-error
+   (package-tests--run-verify-checksums-test t
+     '((sha512 . "invalid")))
+   :type 'bad-checksum))
+
+(ert-deftest package-test-verify-package-checksums-t/missing-fails ()
+  "Verify checksums (t) -- fail on missing."
+  (should-error
+   (package-tests--run-verify-checksums-test t
+     nil)
+   :type 'bad-checksum))
+
+(ert-deftest package-test-verify-package-checksums-t/ignore-unsupported ()
+  "Verify checksums (t) -- ignore unsupported algorithm."
+  (package-tests--run-verify-checksums-test t
+    `((ignore . "not supported")
+      (sha512 . ,package-tests-valid-sha512-checksum))))
+
+(ert-deftest package-test-verify-package-checksums-all ()
+  "Verify checksums (all) -- succeed on valid."
+  (package-tests--run-verify-checksums-test 'all
+    `((md5    . ,package-tests-valid-md5-checksum)
+      (sha256 . ,package-tests-valid-sha256-checksum)
+      (sha512 . ,package-tests-valid-sha512-checksum))))
+
+(ert-deftest package-test-verify-package-checksums-all/invalid-fails ()
+  "Verify checksums (all) -- fail if one checksum is invalid."
+  (should-error
+   (package-tests--run-verify-checksums-test 'all
+     `((md5    . ,package-tests-valid-md5-checksum)
+       (sha256 . "invalid")
+       (sha512 . ,package-tests-valid-sha512-checksum)))
+   :type 'bad-checksum))
+
+(ert-deftest package-test-verify-package-checksums-all/missing-fails ()
+  "Verify checksums (all) -- fail on missing checksums."
+  (should-error
+   (package-tests--run-verify-checksums-test 'all
+     nil)
+   :type 'bad-checksum))
+
+(ert-deftest package-test-verify-package-checksums-all/no-supported-hash-fails ()
+  "Verify checksums (all) -- fail if we have no supported hash."
+  (should-error
+   (package-tests--run-verify-checksums-test 'all
+     '((unsupported . "invalid")))
+   :type 'bad-checksum))
+
+(ert-deftest package-test-verify-package-checksums-all/ignore-unsupported ()
+  "Verify checksums (all) -- succed if one hash algorithm is unsupported.
+If the rest succeed, just ignore the unsupported one."
+  (package-tests--run-verify-checksums-test 'all
+    `((md5    . ,package-tests-valid-md5-checksum)
+      (sha256 . ,package-tests-valid-sha256-checksum)
+      (sha512 . ,package-tests-valid-sha512-checksum)
+      (ignore . "not supported"))))
+
+(ert-deftest package-test-verify-package-size ()
+  (with-temp-buffer
+    (let ((pkg-desc (package-desc-create :size 6)))
+      (insert "123456")
+      (package--verify-package-size pkg-desc)
+      (insert "7")
+      (should-error (package--verify-package-size pkg-desc)))))
+
+(ert-deftest package-test-parse-valid-until-from-buffer ()
+  (with-temp-buffer
+    (insert ";; Valid-Until: 2020-05-01T15:43:35.000Z\n(foo bar baz)")
+    (should (equal (package--parse-valid-until-from-buffer "foo")
+                   '(24236 17319)))))
+
+(ert-deftest package-test-parse-last-updated-from-buffer ()
+  (with-temp-buffer
+    (insert ";; Last-Updated: 2020-05-01T15:43:35.000Z\n(foo bar baz)")
+    (should (equal (package--parse-last-updated-from-buffer "foo")
+                   '(24236 17319)))))
+
+(defun package-tests--parse-last-updated (timestamp)
+  (with-temp-buffer
+    (insert timestamp)
+    (package--parse-last-updated-from-buffer "test")))
+
+(ert-deftest package-test-archive-verify-timestamp ()
+  (let ((a (package-tests--parse-last-updated
+            ";; Last-Updated: 2020-05-01T15:43:35.000Z\n"))
+        (b (package-tests--parse-last-updated
+            ";; Last-Updated: 2020-06-01T15:43:35.000Z\n"))
+        (c (package-tests--parse-last-updated
+            ";; Last-Updated: 2020-07-01T15:43:35.000Z\n")))
+    (should (package--archive-verify-timestamp b nil "foo"))
+    (should (package--archive-verify-timestamp b a "foo"))
+    (should (package--archive-verify-timestamp c a "foo"))
+    (should (package--archive-verify-timestamp c b "foo"))
+    ;; Signal error.
+    (should-error (package--archive-verify-timestamp a b "foo")
+                  :type 'bad-timestamp)
+    (should-error (package--archive-verify-timestamp a c "foo")
+                  :type 'bad-timestamp)
+    (should-error (package--archive-verify-timestamp b c "foo")
+                  :type 'bad-timestamp)
+    (should-error (package--archive-verify-timestamp nil a "foo")
+                  :type 'bad-timestamp)))
+
+(ert-deftest package-test-check-archive-timestamp ()
+  (let ((package-user-dir package-test-data-dir))
+    (with-temp-buffer
+      (insert ";; Last-Updated: 2020-01-01T00:00:00.000Z\n")
+      (package--check-archive-timestamp "older")
+      (package--check-archive-timestamp "missing")
+      (should-error (package--check-archive-timestamp "newer")
+                    :type 'bad-timestamp))))
+
+(ert-deftest package-test-check-archive-timestamp/not-expired ()
+  (let ((package-user-dir package-test-data-dir))
+    (with-temp-buffer
+      (insert ";; Last-Updated: 2020-01-01T00:00:00.000Z\n"
+              ";; Valid-Until: 2999-01-02T00:00:00.000Z\n")
+      (should-not (package--check-archive-timestamp "older")))))
+
+(ert-deftest package-test-check-archive-timestamp/expired ()
+  (let ((package-user-dir package-test-data-dir))
+    (with-temp-buffer
+      (insert ";; Last-Updated: 2020-01-01T00:00:00.000Z\n"
+              ";; Valid-Until: 2020-01-02T00:00:00.000Z\n")
+      (should-error (package--check-archive-timestamp "older")))))
 
 
 ;;; Tests for package-x features.
@@ -684,7 +930,9 @@ Must called from within a `tar-mode' buffer."
                               'single
                               '((:authors ("J. R. Hacker" . "jrh@example.com"))
                                 (:maintainer "J. R. Hacker" . "jrh@example.com")
-                                (:url . "http://doodles.au"))))
+                                (:url . "http://doodles.au"))
+                              nil
+                              nil))
   "Expected contents of the archive entry from the \"simple-single\" package.")
 
 (defvar package-x-test--single-archive-entry-1-4
@@ -693,7 +941,9 @@ Must called from within a `tar-mode' buffer."
                               "A single-file package with no dependencies"
                               'single
                               '((:authors ("J. R. Hacker" . "jrh@example.com"))
-                                (:maintainer "J. R. Hacker" . "jrh@example.com"))))
+                                (:maintainer "J. R. Hacker" . "jrh@example.com"))
+                              nil
+                              nil))
   "Expected contents of the archive entry from the updated \"simple-single\" package.")
 
 (ert-deftest package-x-test-upload-buffer ()