We used to store module runtime and environment pointers in the static
lists Vmodule_runtimes and Vmodule_environments. However, this is
incorrect because these objects have to be kept per-thread. With this
naive approach, interleaving module function calls in separate threads
leads to environments being removed in the wrong order, which in turn
can cause local module values to be incorrectly garbage-collected.
The fix isn't completely trivial: specbinding the lists wouldn't work
either, because then the garbage collector wouldn't find the
environments in other threads than the current ones, again leading to
objects being garbage-collected incorrectly. While introducing custom
pseudovector types would fix this, it's simpler to put the runtime and
environment pointers into the specbinding list as new specbinding
kinds. This works since we need to unwind them anyway, and we only
ever treat the lists as a stack. The thread switching machinery
ensures that the specbinding lists are thread-local, and that all
elements of the specbinding lists in all threads are marked during
garbage collection.
Module assertions now have to walk the specbinding list for the
current thread, which is more correct since they now only find
environments for the current thread. As a result, we can now remove
the faulty Vmodule_runtimes and Vmodule_environments variables
entirely.
Also add a unit test that exemplifies the problem. It interleaves two
module calls in two threads so that the first call ends while the
second one is still active. Without this change, this test triggers
an assertion failure.
* src/lisp.h (enum specbind_tag): Add new tags for module runtimes and
environments.
* src/eval.c (record_unwind_protect_module): New function to record a
module object in the specpdl list.
(do_one_unbind): Unwind module objects.
(backtrace_eval_unrewind, default_toplevel_binding, lexbound_p)
(Fbacktrace__locals): Deal with new specbinding types.
(mark_specpdl): Mark module environments as needed.
* src/alloc.c (garbage_collect): Remove call to 'mark-modules'.
Garbage collection of module values is now handled as part of marking
the specpdl of each thread.
* src/emacs-module.c (Fmodule_load, funcall_module): Use specpdl to
record module runtimes and environments.
(module_assert_runtime, module_assert_env, value_to_lisp): Walk
through specpdl list instead of list variables.
(mark_module_environment): Rename from 'mark_modules'. Don't attempt
to walk though current thread's environments only, since that would
miss other threads.
(initialize_environment, finalize_environment): Don't change
Vmodule_environments variable; environments are now in the specpdl
list.
(finalize_environment_unwind, finalize_runtime_unwind): Make 'extern'
since do_one_unbind now calls them.
(finalize_runtime_unwind): Don't change Vmodule_runtimes variable;
runtimes are now in the specpdl list.
(syms_of_module): Remove Vmodule_runtimes and Vmodule_environments.
* test/data/emacs-module/mod-test.c (Fmod_test_funcall): New test
function.
(emacs_module_init): Bind it.
* test/src/emacs-module-tests.el (emacs-module-tests--variable): New
helper type to guard access to state in a thread-safe way.
(emacs-module-tests--wait-for-variable)
(emacs-module-tests--change-variable): New helper functions.
(emacs-module-tests/interleaved-threads): New unit test.
* src/module-env-28.h: Add field for 'make_interactive' function.
* src/emacs-module.c (Lisp_Module_Function): Add new field holding the
interactive form.
(allocate_module_function): Adapt to structure layout change.
(module_make_interactive, module_function_interactive_form): New
functions.
(initialize_environment): Use them.
* src/eval.c (Fcommandp):
* src/data.c (Finteractive_form): Also handle interactive module
functions.
* test/data/emacs-module/mod-test.c (Fmod_test_identity): New test
function.
(emacs_module_init): Create two interactive module test functions.
* test/src/emacs-module-tests.el (module/interactive/return-t)
(module/interactive/return-t-int, module/interactive/identity):
New unit tests.
* doc/lispref/internals.texi (Module Functions): Document new
function. Rework paragraph about wrapping module functions, as the
example no longer applies.
* etc/NEWS: Document new facility.
The report that they broke macOS was a false alarm, as the
previous commit was also broken (Bug#43152#62).
* src/alloc.c (live_string_holding, live_cons_holding)
(live_symbol_holding):
Count only pointers that point to a struct component,
or are a tagged pointer to the start of the struct.
Exception: for non-bool-vector pseudovectors,
count any pointer past the header, since it’s too much
of a pain to write code for every pseudovector.
(live_float_holding, live_vector_pointer):
New functions, which are similar about counting pointers.
(live_float_p, live_large_vector_holding)
(live_small_vector_pointer, mark_maybe_pointer): Use them.
(mark_maybe_object, mark_maybe_objects): Remove,
and remove all callers; mark_maybe_pointer now suffices.
(mark_objects): New function.
* src/alloc.c (mark_vectorlike, mark_face_cache):
* src/eval.c (mark_specpdl):
* src/fringe.c (mark_fringe_data):
* src/keyboard.c (mark_kboards):
Simplify by using mark_objects.
* src/lisp.h (SAFE_ALLOCA_LISP_EXTRA):
Clear any Lisp_Object arrays large enough to not fit into the stack,
so that GC need not worry about whether they contain objects.
* src/alloc.c (live_string_holding, live_cons_holding)
(live_symbol_holding, live_large_vector_holding)
(live_small_vector_holding):
Go back to old approach of treating every would-be pointer to any
byte in the object (though not to just past the object end) as
addressing the object.
(live_float_p): Require that the would-be float point
to the start of the Lisp_Float, and not anywhere else.
(live_vector_pointer, live_float_holding, mark_objects):
Remove. All uses removed.
(mark_maybe_object, mark_maybe_objects):
Bring back these functions.
* src/lisp.h (SAFE_ALLOCA_LISP_EXTRA): Do not clear the
new slots, as they're now checked via mark_maybe_objects,
not via mark_objects.
* src/alloc.c (mark_maybe_object, mark_maybe_objects): Remove.
(mark_objects): New function.
* src/eval.c (mark_specpdl): Use mark_objects instead of
mark_maybe_objects, since the array now has only valid Lisp objects.
* src/lisp.h (SAFE_ALLOCA_LISP_EXTRA): When allocating a large
array, clear it so that it contains only valid Lisp objects. This
is simpler and safer, and does not hurt performance significantly
on my usual benchmark as the code is executed so rarely.
5280e118c0 (origin/emacs-27) ; * src/xdisp.c (pos_visible_p): Fix las...
bb1a9481c9 Fix posn-at-point at beginning of a display string
0c4b033670 Improve documentation of Info node movement commands
632b0119e1 Add Jansson dependency to Windows Build
dbfcdab837 Unbreak 'reverse-region'
c37de84845 Fix typos and markup in fill column indicator docs
f61bff3ee9 ; * CONTRIBUTE: Clarify the preferences for patch formatting.
368e140660 Avoid crashes in 'defconst'
11e3413cff Fix text about Lisp archives in the Emacs FQ
4c81724675 Don't use 'cl' functions in ELisp manual's examples
Add an initial implementation to support dynamic scope. Arg
parsing/binding it's done using the existing code in use for
bytecode (no ad-hoc code is synthetized for that).
* src/lisp.h (struct Lisp_Subr): Add lambda_list field.
(SUBR_NATIVE_COMPILED_DYNP): New inliner.
* src/alloc.c (mark_object): Update for Add lambda_list field.
* src/eval.c (eval_sub, Ffuncall, funcall_lambda): Handle native
compiled dynamic scope
* src/comp.c (declare_lex_function): Rename from declare_function
and rework.
(declare_function): New function.
(make_subr): Handle daynamic scope
* src/pdumper.c (dump_subr): Update for lambda_list field.
* lisp/emacs-lisp/comp.el (comp-func): Remove args slot.
(comp-func-l, comp-func-d): New classes deriving from `comp-func'.
(comp-spill-lap-function): Rework.
(comp-prepare-args-for-top-level): New function.
(comp-emit-for-top-level, comp-emit-lambda-for-top-level): Make
use of `comp-prepare-args-for-top-level'.
(comp-limplify-top-level): Use `comp-func-l'.
(comp-limplify-function): Emit arg prologue only for dynamic
scoped functions.
(comp-call-optim-form-call): Use `comp-func-l'.
(comp-call-optim, comp-tco): Do not optimize dynamic scoped code.
When closing emacs will inspect all directories from which it loaded
native compilation units. If it finds a ".eln.old" file it will try to
delete it, if it fails that means that another Emacs instance is using it.
When compiling a file we rename the file that was in the output path
in case it has been loaded into another Emacs instance.
When deleting a package we move any ".eln" or ".eln.old" files in the
package folder that we can't delete to `package-user-dir`. Emacs will
check that directory when closing and delete them.
* lisp/emacs-lisp/comp.el (comp--replace-output-file): Function called
from C code to finish the compilation process. It performs renaming of
the old file if necessary.
* lisp/emacs-lisp/package.el (package--delete-directory): Function to
delete a package directory. It moves native compilation units that it
can't delete to `package-user-dir'.
* src/alloc.c (cleanup_vector): Call dispose_comp_unit().
(garbage_collect): Call finish_delayed_disposal_of_comp_units().
* src/comp.c: Restore the signal mask using unwind-protect. Store
loaded native compilation units in a hash table for disposal on
close. Store filenames of native compilation units GC'd in a linked
list to finish their disposal when the GC is over.
(clean_comp_unit_directory): Delete all *.eln.old files in a
directory.
(clean_package_user_dir_of_old_comp_units): Delete all *.eln.old files
in `package-user-dir'.
(dispose_all_remaining_comp_units): Dispose of native compilation
units that are still loaded.
(dispose_comp_unit): Close handle and cleanup directory or arrange for
later cleanup if DELAY is true.
(finish_delayed_disposal_of_comp_units): Dispose of native compilation
units that were GC'd.
(register_native_comp_unit): Register native compilation unit for
disposal when Emacs closes.
* src/comp.h: Introduce cfile member in Lisp_Native_Comp_Unit.
Add declarations of functions that: clean directories of unused native
compilation units, handle disposal of native compilation units.
* src/emacs.c (kill-emacs): Dispose all remaining compilation units
right right before calling exit().
* src/eval.c (internal_condition_case_3, internal_condition_case_4):
Add functions.
* src/lisp.h (internal_condition_case_3, internal_condition_case_4):
Add functions.
* src/pdumper.c (dump_do_dump_relocation): Set cfile to a copy of the
Lisp string specifying the file path.
Check Lisp_Compiled objects better as they’re created,
so that the byte-code interpreter needn’t do the checks
each time it executes them. This improved performance
of ‘make compile-always’ by 1.5% on my platform. Also,
improve the quality of the (still-incomplete) checks, as
this is more practical now that they’re done less often.
* src/alloc.c (make_byte_code): Remove. All uses removed.
(Fmake_byte_code): Put a better (though still incomplete)
check here instead. Simplify by using Fvector instead
of make_uninit_vector followed by memcpy, and by using
XSETPVECTYPE instead of make_byte_code followed by XSETCOMPILED.
* src/bytecode.c (Fbyte_code): Do sanity check and conditional
translation to unibyte here instead of each time the function is
executed.
(exec_byte_code): Omit no-longer-necessary sanity and
unibyte checking. Use SCHARS instead of SBYTES where
either will do, as SCHARS is faster.
* src/eval.c (fetch_and_exec_byte_code): New function.
(funcall_lambda): Use it.
(funcall_lambda, lambda_arity, Ffetch_bytecode):
Omit no-longer-necessary sanity checks.
(Ffetch_bytecode): Add sanity check if actually fetching.
* src/lisp.h (XSETCOMPILED): Remove. All uses removed.
* src/lread.c (read1): Check byte-code objects more thoroughly,
albeit still incompletely, and do translation to unibyte here
instead of each time the function is executed.
(read1): Use XSETPVECYPE instead of make_byte_code.
(read_vector): Omit no-longer-necessary sanity check.
backtrace_eval_unrewind is used to temporarily reverse
let-bindings (it's called with a positive argument to reverse
bindings, and then a negative argument to re-apply them) by
backtrace--locals and backtrace-eval. For the SPECPDL_LET_DEFAULT and
SPECPDL_LET_LOCAL cases (which occur for let-bindings on buffer-local
variables), the code calls Fdefault_value and Fbuffer_local_value on
the symbol.
For symbols which are unbound at top-level, the first (with positive
argument) call to backtrace_eval_unrewind will set the symbol's value
to unbound (putting the current value in the specpdl's "old value"
slot). On the second (with negative argument) call,
backtrace_eval_unrewind attempts to retrieve the symbol's value with
Fdefault_value or Fbuffer_local_value, but that raises a void-variable
signal. This interrupts the restoration of the let-bindings, so any
other variables more recent on the stack will now have the wrong
value.
* src/data.c (default_value): Make non-static.
* src/lisp.h: Declare it.
* src/eval.c (backtrace_eval_unrewind): Replace the calls to
Fdefault_value and Fbuffer_local_value with default_value and
buffer_local_value, respectively. The latter do exactly the same as
the former, except if the symbol's value is Qunbound they just return
it instead of signaling void-variable.
Occasionally, loading cl-generic.el from source requires
max_specpdl_size > 1600 when bootstrapping, and thus fails.
In any case we are very close to the limit.
* src/eval.c (init_eval_once): Raise max_specpdl_size to 1800.
* doc/lispref/variables.texi (Local Variables): Update docs.
Problem reported by Pip Cet in:
https://lists.gnu.org/r/emacs-devel/2019-07/msg00520.html
* src/alloc.c (inhibit_garbage_collection): Use new function.
(allow_garbage_collection): Accept intmax_t, not pointer.
* src/eval.c (default_toplevel_binding, do_one_unbind)
(backtrace_eval_unrewind, Fbacktrace__locals, mark_specpdl):
Support SPECPDL_UNWIND_INTMAX.
(record_unwind_protect_excursion): New function.
* src/lisp.h (enum specbind_tag): New constant SPECPDL_UNWIND_INTMAX.
(union specbinding): New member unwind_intmax.
Without this patch, (capitalize "x") can alter the match data,
which is not what users expect. Problem found by running
morse-tests-unnato-region in a stripped-down Emacs.
Perhaps ‘load’ should also save and restore the match data?
That would be a simpler fix, though arguably incompatible.
* src/lread.c (save_match_data_load): New function.
* src/chartab.c (uniprop_table):
* src/doc.c (reread_doc_file):
* src/eval.c (Fautoload_do_load):
* src/fns.c (Frequire): Use it.
Point out that 'function' quoting is beneficial also for symbols.
* src/eval.c (function): Enhance docstring.
* doc/lispref/functions.texi (Anonymous Functions): Improve
documentation.
Avoid the abuse of (eval `(defvar ...)) which tends to end up
adding redundant entries in `load-history`, as discussed in
https://lists.gnu.org/r/help-gnu-emacs/2019-03/msg00237.html
(custom-initialize-default): Don't add to load-history.
(custom-declare-variable): Use internal--define-uninitialized-variable
and only add the var to load-history once. Do it before calling
`initialize` so the special-variable-p flag is set.
* src/eval.c (Finternal__define_uninitialized_variable): New function.
(Fdefvar, Fdefconst): Use it.
(syms_of_eval): Defsubr' it.
Stefan Monnier pointed out examples like (funcall `(closure
,(let ((cycle (list nil))) (setcdr cycle cycle)) () a)),
where the user can set Vinternal_interpreter_environment
indirectly.
* src/eval.c (Fsetq): Revert recent change, going back to Fassq.
* src/eval.c (eval_sub): Check whether Fassq returns Qnil,
not whether it returns a cons, as NILP is faster than CONSP
nowadays. Remove incorrect comment “only original_fun and
original_args have values that will be used below”; instead,
move declarations around so that the set of variables with
useful values is obvious.
In some cases, we never specbind internal objects, so they don't have
to be symbols. Rather than using DEFSYM/DEFVAR and then uninterning
the symbols, use plain static variables. Call staticpro for all of
them, to protect them from the garbage collector.
* src/eval.c (syms_of_eval): Use a static variable for
Qcatch_all_memory_full.
* src/emacs-module.c (syms_of_module): Use static variables for
Vmodule_refs_hash, Vmodule_runtimes, and Vmodule_environments.
