diff --git a/mps/manual/source/guide/advanced.rst b/mps/manual/source/guide/advanced.rst index 46ccdf5768e..e62b7842414 100644 --- a/mps/manual/source/guide/advanced.rst +++ b/mps/manual/source/guide/advanced.rst @@ -754,6 +754,9 @@ is detected, we can see this happening: :ref:`topic-weak`, :ref:`pool-awl`. +.. index:: + single: Scheme; global symbol table + Global symbol table ------------------- @@ -852,3 +855,60 @@ detected by the scan method, we can see when symbols are dying: Here, the symbols ``b``, ``c`` and ``d`` died, but ``a`` was kept alive by the reference from the environment. + + +.. index:: + single: Scheme; segregation + +.. _guide-advanced-segregation: + +Segregation of objects +---------------------- + +When objects of different types have different properties (different +sizes, lifetimes, references, layouts) it makes sense to segregate +them into pools of appropriate classes. + +For example, the toy Scheme interpreter has a mixture of object types, +some of which contain references to other objects (for example, pairs) +that must be :term:`scanned `, and some of which do not (for +example, strings). If the :term:`leaf objects` are segregated into a +pool of an appropriate class, the cost of scanning them can be +avoided. + +Here the appropriate class is :ref:`pool-amcz`, and the necessary code +changes are straightforward. First, global variables for the new pool +and its :term:`allocation point`:: + + static mps_pool_t leaf_pool; /* pool for leaf objects */ + static mps_ap_t leaf_ap; /* allocation point for leaf objects */ + +Second, the leaf objects must be allocated on ``leaf_ap`` instead of +``obj_ap``. And third, the pool and its allocation point must be created:: + + /* Create an Automatic Mostly-Copying Zero-rank (AMCZ) pool to + manage the leaf objects. */ + res = mps_pool_create(&leaf_pool, + arena, + mps_class_amcz(), + obj_fmt, + obj_chain); + if (res != MPS_RES_OK) error("Couldn't create leaf pool"); + + /* Create allocation point for leaf objects. */ + res = mps_ap_create(&leaf_ap, leaf_pool); + if (res != MPS_RES_OK) error("Couldn't create leaf objects allocation point"); + +Note that the new pool shared a :term:`generation chain` with the old +pool. This is important, because the leaf objects live and die along +with the non-leaf objects of similar ages. + +As an initial step in making this change, the new pool uses the same +:term:`object format`. However, we normally wouldn't stop there: we'd +take advantage of the segregation to simplify the scanning of the +objects that have been left behind. + +.. topics:: + + :ref:`pool-amcz`. + diff --git a/mps/manual/source/pool/amc.rst b/mps/manual/source/pool/amc.rst index 8cdb7cc24c9..99a4e49778e 100644 --- a/mps/manual/source/pool/amc.rst +++ b/mps/manual/source/pool/amc.rst @@ -13,10 +13,10 @@ AMC (Automatic Mostly-Copying) ============================== **AMC** is a general-purpose :term:`automatically managed ` :term:`pool class`. This is the most "advanced" -pool class in the MPS, intended for the majority of objects in the -client program. Use this pool class unless you need a particular -feature that it doesn't provide. +memory management>` :term:`pool class`. This is the most mature pool +class in the MPS, intended for the majority of objects in the client +program. Use this pool class unless you need a particular feature that +it doesn't provide. "Mostly Copying" means that it uses :term:`copying garbage collection` except for blocks that are :term:`pinned ` by @@ -29,10 +29,7 @@ following tendencies will be efficiently exploited by an AMC pool: - most objects die young; -- objects that don't die young will live a long time; - -- most references are "backwards in time" (from younger objects to - older objects). +- objects that don't die young will live a long time. In the pool's :term:`generation chain`, specify the capacity and mortality of generations 0 to *n*\−1. Survivors from generation *n*\−1 @@ -153,9 +150,6 @@ AMC introspection on :term:`dead` objects that the collector has not recycled or has been unable to recycle. - The function ``f`` may not allocate memory or access any - automatically-managed memory except within ``object``. - .. note:: There is no equivalent function for other pool classes, but diff --git a/mps/manual/source/pool/amcz.rst b/mps/manual/source/pool/amcz.rst index 355f24736ee..6ee47671005 100644 --- a/mps/manual/source/pool/amcz.rst +++ b/mps/manual/source/pool/amcz.rst @@ -20,7 +20,10 @@ It is otherwise indentical to :ref:`pool-amc`. AMCZ is intended for "simple" objects like numbers, characters, and strings. Segregating these objects into one or more AMCZ pools avoids the cost of scanning them that would be incurred if they were -interleaved in a pool with objects containing references. +interleaved in a pool with objects containing references. It may also +simplify the scanning of the objects that are left behind. + +See :ref:`guide-advanced-segregation` for an example. .. index:: diff --git a/mps/manual/source/pool/awl.rst b/mps/manual/source/pool/awl.rst index 4adb55b6516..8ed131653da 100644 --- a/mps/manual/source/pool/awl.rst +++ b/mps/manual/source/pool/awl.rst @@ -35,6 +35,9 @@ AWL allows the implementation to splat the value slot at the same time that the weak key slot is splatted. (Or the other way around for weak-value tables.) See :ref:`pool-awl-dependent`. +See :ref:`guide-advanced-weak` in the :ref:`guide-advanced` section of +the user guide for a detailed example of using this pool class. + .. note:: AWL is the only pool in the open source MPS that allows its diff --git a/mps/manual/source/pool/intro.rst b/mps/manual/source/pool/intro.rst index 685eb7504c3..1d0b7cec744 100644 --- a/mps/manual/source/pool/intro.rst +++ b/mps/manual/source/pool/intro.rst @@ -1,3 +1,6 @@ +.. index:: + single: pool class; choosing + .. _pool-choose: Choosing a pool class @@ -13,7 +16,9 @@ of pools, choosing the most appropriate pool class for each. Pool classes can differ in many ways not considered here: speed, vulnerability to fragmentation, control overhead, and so on. This procedure gives you a decent recommendation, but an expert in the - MPS might be able to make a better one. + MPS might be able to make a better recommendation. And if no pool + class in the open source MPS exactly matches your needs, then it + is possible to develop new pool classes. See :ref:`pool-writing`. First, answer these questions about your data: @@ -65,6 +70,9 @@ no *any* weak :ref:`pool-mvt` [1]_ ``_ +.. index:: + single: pool class; table of properties + .. _pool-properties: Pool class properties @@ -96,7 +104,6 @@ Allocations fixed or variable in size? var var var var var Alignment? [5]_ conf conf conf conf conf [6]_ [6]_ [7]_ [6]_ conf Dependent objects? [8]_ no --- no yes --- --- --- --- --- no May use remote references? [9]_ no --- no no --- --- --- --- --- no -Ambiguous references keep blocks alive? no no no no no --- --- --- --- no Blocks are automatically managed? [10]_ yes yes yes yes yes no no no no no Blocks are manually managed? [10]_ no no no no no yes yes yes yes yes Blocks are scanned? [11]_ yes no yes yes no no no no no yes @@ -172,6 +179,11 @@ Blocks may belong to format auto-header? yes yes yes yes yes to the block. +.. index:: + single: pool class; writing + +.. _pool-writing: + Writing a new pool class ======================== diff --git a/mps/manual/source/pool/lo.rst b/mps/manual/source/pool/lo.rst index 822b373aeb3..20e02c82a54 100644 --- a/mps/manual/source/pool/lo.rst +++ b/mps/manual/source/pool/lo.rst @@ -33,8 +33,9 @@ that needs to be passed to an operating system I/O function. has finished (for example, by keeping a reference to the buffer in a :term:`root` or a :term:`scanned ` object). -For leaf objects that can move and be protected, consider -:ref:`pool-amcz` instead. +If LO is used to allocate large numbers of small objects, the garbage +collection performance will degrade. For leaf objects that can move +and be protected, it is better to use :ref:`pool-amcz` instead. .. index:: diff --git a/mps/manual/source/pool/mvt.rst b/mps/manual/source/pool/mvt.rst index cf58cab9e05..6fd51db85d9 100644 --- a/mps/manual/source/pool/mvt.rst +++ b/mps/manual/source/pool/mvt.rst @@ -105,7 +105,7 @@ MVT interface :: - #include "mpscmv2.h" + #include "mpscmvt.h" .. c:function:: mps_class_t mps_class_mvt(void) @@ -186,7 +186,7 @@ MVT introspection :: - #include "mpscmv2.h" + #include "mpscmvt.h" .. c:function:: size_t mps_mvt_free_size(mps_pool_t pool) diff --git a/mps/manual/source/topic/debugging.rst b/mps/manual/source/topic/debugging.rst index 9e149f2ace9..4646818c966 100644 --- a/mps/manual/source/topic/debugging.rst +++ b/mps/manual/source/topic/debugging.rst @@ -11,18 +11,30 @@ Debugging pools =============== -Several :term:`pool classes` have debugging counterparts that provide -two features that are useful for debugging: +Several :term:`pool classes` have debugging counterparts: + +================= ============================== +Pool class Debugging counterpart +================= ============================== +:ref:`pool-ams` :c:func:`mps_class_ams_debug` +:ref:`pool-mv` :c:func:`mps_class_mv_debug` +:ref:`pool-mvff` :c:func:`mps_class_mvff_debug` +================= ============================== + +These debugging pool classes provide two features that are useful for +debugging: * .. index:: single: debugging; fencepost single: fencepost :dfn:`fenceposts` are patterns of data that are written before and - after each allocated block. These patterns can be checked, for - example when the block is deallocated, to see that they are - unchanged. This helps detect underwriting and :term:`overwriting - errors`. + after each allocated block. In :term:`manually managed ` pools, fenceposts are checked when the block is + deallocated, to see that they are unchanged. This helps detect + underwriting and :term:`overwriting errors`. Fenceposts for all + objects in a pool are checked when the pool is destroyed, and can be + checked at any time by calling :c:func:`mps_pool_check_fenceposts`. * .. index:: single: debugging; free space splatting @@ -32,9 +44,11 @@ two features that are useful for debugging: of data. If the pattern is designed so that it does not resemble a live object (and if code checks the consistency of its data structues), then this helps to detect :term:`dangling pointer` - dereferences. The pattern can also be checked, for example just - before allocation, or when a block of memory is released from the - pool to the arena, to see that it is unchanged. + dereferences. The pattern is checked just before allocation, and + when a block of memory is released from the pool to the arena, to + see that it is unchanged. All free space in a pool can be checked + for the pattern at any time by calling + :c:func:`mps_pool_check_free_space`. The :term:`client program` specifies templates for both of these features via the :c:type:`mps_pool_debug_option_s` structure. This diff --git a/mps/manual/source/topic/pattern.rst b/mps/manual/source/topic/pattern.rst index fbd53b65ed8..4d798c00b78 100644 --- a/mps/manual/source/topic/pattern.rst +++ b/mps/manual/source/topic/pattern.rst @@ -12,8 +12,7 @@ Allocation patterns An :dfn:`allocation pattern` is a hint to the MPS to expect a particular pattern of allocation on an :term:`allocation point`. The -MPS may use this hint to schedule its decisions as to when and what to -collect. +MPS may use this hint to schedule more effective garbage collection. There are two allocation patterns, :c:func:`mps_alloc_pattern_ramp` and :c:func:`mps_alloc_pattern_ramp_collect_all`. @@ -41,6 +40,12 @@ and :c:func:`mps_alloc_pattern_ramp_collect_all`. code` if the allocation pattern is not supported by the allocation point. + .. note:: + + It is harmless to call :c:func:`mps_ap_alloc_pattern_begin` + even if it isn't supported by the allocation point. The + pattern is simply ignored in that case. + If :c:func:`mps_ap_alloc_pattern_begin` is used multiple times on the same allocation point without intervening calls to :c:func:`mps_ap_alloc_pattern_end`, the calls match in a @@ -64,7 +69,13 @@ and :c:func:`mps_alloc_pattern_ramp_collect_all`. Returns :c:macro:`MPS_RES_OK` if the period of allocation was successfully ended, or :c:macro:`MPS_RES_FAIL` if there was no - corresponding call to :c:func:`mps_ap_alloc_pattern_begin`. + matching call to :c:func:`mps_ap_alloc_pattern_begin`. Calls match + in a stack-like way, outermost and innermost: that is, allocation + patterns may nest, but not otherwise overlap. + + Some allocation patterns may additionally support overlap: if so, + the documentation for the individual pattern types will specify + this. .. c:function:: mps_res_t mps_ap_alloc_pattern_reset(mps_ap_t ap) diff --git a/mps/manual/source/topic/telemetry.rst b/mps/manual/source/topic/telemetry.rst index b3c398cb5bd..43bb9770807 100644 --- a/mps/manual/source/topic/telemetry.rst +++ b/mps/manual/source/topic/telemetry.rst @@ -46,6 +46,28 @@ demonstration of :term:`Lisp` in an appendix to his paper thought we were victims of a practical joker. + +.. index:: + single: telemetry; utilities + +Telemetry utilities +------------------- + +The telemetry system relies on two utility programs: + +* :program:`mpseventcnv` decodes the machine-dependent binary event + stream into a portable text format. It must be compiled for the same + architecture as the MPS-linked program whose event stream it + decodes. + +* :program:`mpseventsql` takes the output of :program:`mpseventcnv` + and loads it into a SQLite database for further analysis. + +You must build and install these programs as described in +:ref:`guide-build`. + + + .. index:: single: telemetry; example single: Scheme; telemetry @@ -194,7 +216,7 @@ telemetry feature. In addition, the following environment variable controls the behaviour of the :program:`mpseventsql` program. -.. envvar:: MPS_EVENT_DATABASE +.. envvar:: MPS_TELEMETRY_DATABASE The name of a SQLite database file that will be updated with the events from the decoded telemetry stream, if it is not specified @@ -271,7 +293,7 @@ be loaded into a SQLite database for further analysis by running events from the decoded telemetry stream specified by the ``-l`` option. The database will be created if it does not exist. If not specified, the file named by the environment variable - :envvar:`MPS_EVENT_DATABASE` is used; if this variable is not + :envvar:`MPS_TELEMETRY_DATABASE` is used; if this variable is not assigned, ``mpsevent.db`` is used. Updating a database with events from a file is idempotent unless @@ -359,6 +381,14 @@ Telemetry interface interactive tool require access to the telemetry stream. You could even try calling this from a debugger after a problem. + .. note:: + + Unless all :term:`arenas` are properly destroyed (by calling + :c:func:`mps_arena_destroy`), there are likely to be unflushed + telemetry events when the program finishes. So in the case of + abnormal program termination such as a fatal exception, you + may want to call :c:func:`mps_telemetry_flush` explicitly. + .. index:: pair: telemetry; labels @@ -390,9 +420,10 @@ Typical uses of telemetry labels include: .. note:: - The ``User`` event category must be turned on in the + If the ``User`` event category is not turned on in the :term:`telemetry filter` (via :c:func:`mps_telemetry_control`) - before calling this function. + then the string is not sent to the telemetry stream. A label + is still returned in this case, but it is useless. .. c:function:: void mps_telemetry_label(mps_addr_t addr, mps_word_t label) @@ -410,6 +441,6 @@ Typical uses of telemetry labels include: .. note:: - The ``User`` event category must be selected in the + If the ``User`` event category is not turned on in the :term:`telemetry filter` (via :c:func:`mps_telemetry_control`) - before calling this function. + then calling this function has no effect. diff --git a/mps/manual/source/topic/weak.rst b/mps/manual/source/topic/weak.rst index 548005c273d..b77e67764bc 100644 --- a/mps/manual/source/topic/weak.rst +++ b/mps/manual/source/topic/weak.rst @@ -13,7 +13,7 @@ Weak references A :dfn:`weak reference` is a :term:`reference` that does not keep the block it refers to :term:`alive `. -The MPS supports weak references only: +The open source MPS supports weak references only: 1. in :term:`roots` that are registered with :term:`rank` :c:func:`mps_rank_weak`; @@ -22,6 +22,11 @@ The MPS supports weak references only: class :ref:`pool-awl` that was created with :term:`rank` :c:func:`mps_rank_weak`. +.. note:: + + If you need more general handling of weak references, + :ref:`contact us `. + When the MPS determines that a block is only kept alive by one or more weak references, it may choose to :term:`splat` those references by replacing them with null pointers when they are :term:`fixed`. When