We didn't add an integration for Imenu because we aren't sure what
should it look like. Now we have a pretty good idea. All the major
modes copy-paste the two Imenu functions and tweaks them in a standard
way. With the addition of treesit-defun-type-regexp and
treesit-defun-name-function, now is a good time to standardize Imenu
integration.
In the next commit we update all the major modes to use this
integration.
* doc/lispref/modes.texi (Imenu): Add manual.
* doc/lispref/parsing.texi (Tree-sitter major modes): Update manual.
* lisp/treesit.el (treesit-simple-imenu-settings): New varaible.
(treesit--simple-imenu-1)
(treesit-simple-imenu): New functions.
(treesit-major-mode-setup): Setup Imenu.
Current implementation of treesit--things-around only searches forward
for REGEXP and go up the tree until it finds a valid thing, if nothing
matches it gives up. This makes it sometimes miss defuns. The new
implementation tries multiple times (of search forward + go up) until
it exhausts all possible defun nodes.
* lisp/treesit.el (treesit--things-around): New implementation.
(treesit--navigate-defun): Refactor to use treesit-node-top-level to
simplify code, and add some guards in the predicate function.
* test/src/treesit-tests.el:
(treesit--ert-defun-navigation-elixir-program): New variable.
(treesit-defun-navigation-nested-4): New test.
* lisp/treesit.el (treesit-node-top-level): Now it can accept a
predicate function. Add an optional argument INCLUDE-NODE.
(treesit-parent-until): Add an optional argument INCLUDE-NODE.
* doc/lispref/parsing.texi (Tree-sitter major modes): Replace vindex
with cross-reference to modes.texi. Add manual entry for
treesit-defun-type-regexp.
* lisp/treesit.el (treesit-defun-type-regexp): Use pred in docstring
since we use pred everywhere else.
Two new functions, treesit-beginning/end-of-thing. And
treesit-thing-at-point's signature changes.
* lisp/treesit.el (treesit-block-type-regexp): New variable.
(treesit-beginning-of-thing)
(treesit-end-of-thing): Generalized from
treesit-beginning/end-of-defun.
(treesit-beginning-of-defun)
(treesit-end-of-defun): Use the new functions.
(treesit-thing-at-point): Accept PATTERN rather than REGEXP and PRED.
(treesit-defun-at-point): Adjust for the new signature of
treesit-thing-at-point.
Change the "defun" in some functions (e.g. treesit--defuns-around) to
"thing". Add a function treesit-thing-at-point.
* lisp/treesit.el (treesit--thing-unpack-pattern): New subroutine.
(treesit-beginning-of-defun)
(treesit-end-of-defun): Use new function treesit--navigate-thing.
(treesit--defuns-around): Generalize into treesit--thing-around.
(treesit--top-level-defun): Generalize into treesit--top-level-thing.
(treesit--navigate-defun): Generalize into treesit--navigate-thing.
(treesit-thing-at-point): Generalized from treesit-defun-at-point.
(treesit-defun-at-point): Use treesit-thing-at-point to do tht work.
1. We now have treesit-defun-name, powered by
treesit-defun-name-function.
2. We now have treesit-add-log-current-defun, which powers
add-log-current-defun.
3. c-ts-mode updates its code to take advantage of these new features.
4. Manual updates.
* doc/lispref/parsing.texi (Tree-sitter major modes): Add manual for
new functions.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--defun-name): New function.
(c-ts-mode--imenu-1): Extract out into c-ts-mode--defun-name.
(c-ts-base-mode): Setup treesit-defun-name-function.
* lisp/treesit.el (treesit-defun-name-function)
(treesit-add-log-defun-delimiter): New variables.
(treesit-defun-at-point)
(treesit-defun-name): New functions.
(treesit-major-mode-setup): Setup add-log-current-defun-function.
If PARSER-OR-LANG is nil, it makes more sense to guess the language at
point by treesit-language-at than to simply use the first parser in
the parser list.
* doc/lispref/parsing.texi (Retrieving Nodes): Update manual.
* lisp/treesit.el (treesit-node-at)
(treesit-node-on): Guess language at point. Update docstring.
(treesit-buffer-root-node): Update docstring.
Now indent and filling works like in c-mode. The only noticeable
missing piece is that the "*/" is not attached to the last sentence
when filling. c-mode does it by replacing whitespaces between the
"*/" and the end of the last sentence with xxx, fill it, then change
the xxx back. I don't know if we should do that in c-ts-mode's filling.
* doc/lispref/modes.texi (Parser-based Indentation): Add new preset.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--indent-styles): Add new
indent rule.
(c-ts-mode--fill-paragraph): New function.
(c-ts-base-mode): Setup paragraph-start, adaptive-fill, etc.
* lisp/treesit.el (treesit-simple-indent-presets): Add new preset.
Comment indentation should use a adaptive-fill-based indent, rather
than comment-start-skip.
Also remove manual description of removed variables in treesit.el and
add documentation for n-p-gp upon request.
* doc/lispref/modes.texi (Parser-based Indentation)
* lisp/treesit.el (treesit-simple-indent-presets): Add n-p-gp, remove
treesit-comment-start/end, remove comment-start-skip.
* doc/lispref/parsing.texi (Tree-sitter major modes): Remove
treesit-comment-start/end.
Because they are not used anymore, after the new defun navigation is
installed.
* lisp/treesit.el (treesit-defun-prefer-top-level)
(treesit--defun-maybe-top-level): Remove.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--defun-valid-p)
(c-ts-mode--defun-skipper): New functions.
(c-ts-base-mode): Setup defun navigation.
* lisp/progmodes/sh-script.el (bash-ts-mode): Setup defun navigation.
* lisp/treesit.el (treesit-beginning-of-defun)
(treesit-end-of-defun): Change to new implementation, which is
intended to be used as commands.
(treesit-major-mode-setup): Setup remap for beginning/end-of-defun
commands.
This new set of functions (and tests) should eliminate
defun-navigation bugs and limitations we currently have. This commit
doesn't change any existing bahavior: treesit-beginning/end-of-defun
and friends are unchanged. The plan is to later switch gear and
replace the current functions with the new ones introduced in this
change.
This is a relatively big change, but I've setup a comprehensive test,
and it should fix current bugs, so I think it's ok to put it on the
release branch.
The gist of the new navigation is to use treesit--defuns-around to
find the previous sibling defun, next sibling defun, and the parent
defun, then use this information to move to previous/next
beginning/end of defun in treesit--navigate-defun.
I also added comprehensive testing that tests all four possible
operations (prev-beg, next-beg, prev-end, next-end) starting at all
possible positions (between two sibling defuns, inside a sibling
defun, etc).
* lisp/treesit.el (treesit-defun-type-regexp): Expand definition to
allow (REGEXP . FILTER). Old functions don't support this, but it
should be fine since we are soon replacing them.
(treesit-defun-tactic)
(treesit-defun-skipper): New variables.
(treesit-default-defun-skipper)
(treesit--defuns-around)
(treesit--top-level-defun)
(treesit--navigate-defun): New functions.
* test/src/treesit-tests.el (treesit--ert-insert-and-parse-marker)
(treesit--ert-collect-positions)
(treesit--ert-test-defun-navigation): New helper functions.
(treesit--ert-defun-navigation-python-program)
(treesit--ert-defun-navigation-js-program)
(treesit--ert-defun-navigation-bash-program)
(treesit--ert-defun-navigation-nested-master): New variables.
(treesit-defun-navigation-nested-1)
(treesit-defun-navigation-nested-2)
(treesit-defun-navigation-nested-3)
(treesit-defun-navigation-top-level): New tests.
* lisp/treesit.el (treesit-explorer-anonymous-node)
(treesit-explorer-field-name): New face.
(treesit--explorer-draw-node): Use the new faces.
(treesit-explore-mode): Change playground to explorer.
Before this change Ftreesit_query_capture doesn't convert character
position to byte position for BEG and END parameters. I observed
fontification issue in css files but couldn't figure out why, now I
know :-)
I decide to keep treesit--font-lock-query-expand-range, since it might
provide a escape hatch for problems we discover in the future, and it
should be very cheap so no downside of keeping it.
* lisp/textmodes/css-mode.el (css-ts-mode): Stop setting
treesit--font-lock-query-expand-range.
* lisp/treesit.el (treesit--font-lock-query-expand-range): Update
docstring.
* src/treesit.c (Ftreesit_query_capture): Convert BEG and END to byte
position. Also added parentheses wround "beg_byte - visible_beg" in
the call to ts_query_cursor_set_byte_range (i.e., style change).
This also fixes fontification problem with c-ts-mode--fontify-defun.
Now treesit-fontify-with-override clips the fontification region for
the user, so no need for (max start node-start) shenanigans anymore.
More importantly it doesn't fontify unless the region between
node-start and node-end intersects with the region between start and
end, which fixes the problem with c-ts-mode--fontify-defun.
* lisp/treesit.el (treesit-fontify-with-override): Add optional
parameter BOUND-START and BOUND-END. Wrap the function body in a
when-form.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--fontify-declarator)
(c-ts-mode--fontify-variable)
(c-ts-mode--fontify-defun)
(c-ts-fontify-error)
* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string): Use the
new signature.
treesit-comment-start/end is unnecessary because of
comment-start/end-skip, so they should be removed.
Cleanup and set comment-start/end-skip for tree-sitter C-like major
modes.
I replaced the [ \t]* part in comment-start-skip with (syntax
whitespace), which is what comment-end-skip uses. I also added
grouping in comment-start-skip to match that of comment-end-skip.
* lisp/progmodes/c-ts-mode.el (c-ts-mode)
(c++-ts-mode)
* lisp/progmodes/csharp-mode.el (csharp-ts-mode)
* lisp/progmodes/java-ts-mode.el (java-ts-mode)
* lisp/progmodes/js.el (js-ts-mode)
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-mode): Setup
comment-start/end-skip.
* lisp/treesit.el (treesit-comment-start)
(treesit-comment-end): Remove variables.
(treesit-simple-indent-presets): Use comment-start/end-skip instead.
This replaces font-lock-maximum-decoration and allows us to disable
the busiest fontification level by default.
* lisp/treesit.el (treesit-font-lock-level): New variable.
(treesit-font-lock-feature-list)
(treesit-font-lock-settings): Change docstring.
(treesit-font-lock-recompute-features): Use the new variable.
Before this change, treesit-beginning-of-defun skips nested defuns.
Now user can decide whether to skip nested defuns.
* lisp/treesit.el (treesit-search-forward-goto): Improve docstring.
(treesit-defun-prefer-top-level): New variable.
(treesit--defun-maybe-top-level): New function.
(treesit-beginning-of-defun)
(treesit-end-of-defun): Use treesit--defun-maybe-top-level.
