scope-capture

If there's something strange, in your neighborhood, who you gonna call? Scope capture!

If there's something weird, and it don't look good, who you gonna call? Scope capture!

[vvvvalvalval/scope-capture "0.3.3"]

This library eases REPL-based development, by providing macros which help you save and restore the local environment of a piece of code with minimal effort.

Project status: beta quality. On the other hand, you typically will only use it in your development environment, so there's little risk in adoption.

Demo video:

Talk at Clojure Days 2018:

• Basic Usage
• Tutorial
• Tips and Tricks

Rationale

This library is designed to support the programming / debugging methodology advocated by Stuart Halloway in this blog post: REPL Debugging: No Stacktrace Required, which consists of:

1. recreating the environment in which a piece of code runs (using defs)
2. evaluating forms in the code buffer to narrow down the cause of the problem

What the blog post does not mention is that oftentimes, this first step (recreating the local environment) can get very tedious and error-prone; especially when the values of the environment are difficult to fabricate (HTTP requests, database connections, etc.), which can be the case for online programs such as web servers, or if you don't have a keen knowledge of the project.

scope-capture alleviates this pain by:

• providing macros that let you save snapshots of the local environment on the fly: sc.api/spy (which additionally saves the evaluated wrapped expression - or the resulting error, which you can inspect using sc.api/saved-value) and sc.api/brk (which acts as a breakpoint, blocking the flow of the program until you choose to release it from the REPL using sc.api/loose!, possibly with a value which supersedes the evaluation of the wrapped expression using sc.api/loose-with! or sc.api/loose-with-ex!)
• providing macros that let you restore these snapshots from the REPL: sc.api/defsc (recreates the environment with global vars, i.e by def-ing the local names) and sc.api/letsc (recreates the environment with locals, i.e by let-ing the local names)

Benefits

As a consequence, to reproduce the runtime context of a code expression, you only need to get it to execute once (not necessarily from the REPL). This makes for:

1. Easier debugging, as you can immediately focus on searching for the cause of the bug
2. Easier project onboarding, especially for beginners. For someone new to project, and even more so someone new to Clojure, manually fabricating the context of a piece of code at the REPL can be a daunting task, as it requires a relatively comprehensive knowledge of the flow of values through the program. This library lets you do that in a completely mechanical and uninformed way.
3. Easier exploration. Because it lowers the barrier to experimentation, this library can be also useful for other tasks than debugging and development, such as running one-off queries as variants of existing ones, or just understading how a project works.

Features

• Recording / logging the runtime scope and evaluation of an expression: sc.api/spy
• Re-creating a recorded scope: with let-bound locals: sc.api/letsc / with global Vars: sc.api/defsc/ with a sub-REPL: sc.repl/ep-repl
• Accessing recorded information: sc.api/ep-info, sc.api/cs-info
• Suspending and resuming execution (similar to breakpoints): sc.api/brk, sc.api/loose, sc.api/loose-with, sc.api/loose-with-err
• Cleaning up after yourself: sc.api/undefsc, sc.api/dispose!, sc.api/disable!
• Customizing: sc.api/spy-emit, sc.api/brk-emit, sc.api.logging/register-cs-logger
• Artificially creating scopes: sc.api/save-ep

For nREPL integration, see also the scope-capture-nrepl companion library.

Installation

With both Leiningen and Boot, it's better not to include scope-capture in your project dependencies but rather in your local environment. Here's how to do it :

Leiningen

Add the following to the :user profile in ~/.lein/profiles.clj:

:dependencies [[vvvvalvalval/scope-capture "0.3.3"]]
:injections [(require 'sc.api)]

Boot

Using a $BOOT_HOME/profile.boot (usually ~/.boot/profile.boot) file:

(set-env! :dependencies #(conj % '[vvvvalvalval/scope-capture "0.3.2"]))
(require 'sc.api)

Docs and API docs

Usage

(See also the detailed tutorial)

(require 'sc.api)

Assume you need to debug a function with a bunch of locals:

(def my-fn 
  (let [a 23 
        b (+ a 3)]
    (fn [x y z]
      (let [u (inc x)
            v (+ y z u)]
        (* (+ x u a)
          ;; Insert a `spy` call in the scope of these locals
          (sc.api/spy
            (- v b)))
        ))))
=> #'sc.lab.example/my-fn

When compiling the function, you will see a log like the following:

SPY <-3> /Users/val/projects/scope-capture/lab/sc/lab/example.cljc:52 
  At Code Site -3, will save scope with locals [a b x y z u v]

Now call the function:

(my-fn 3 4 5)
=> -390

You will see a log like the following:

SPY [7 -3] /Users/val/projects/scope-capture/lab/sc/lab/example.cljc:52 
  At Execution Point 7 of Code Site -3, saved scope with locals [a b x y z u v]
SPY [7 -3] /Users/val/projects/scope-capture/lab/sc/lab/example.cljc:52 
(- v b)
=>
-13

You can now use the letsc macro to recreate the scope of your spy call at the previous execution:

(sc.api/letsc 7
  [a b u v x y z])
=> [23 26 4 13 3 4 5]

(sc.api/letsc 7
  (+ x u a))
=> 30  

You can also use defsc to recreate the scope by def-ing Vars, which is more convenient if you're using the 'evaluate form in REPL' command of your editor:

(sc.api/defsc 7)
=> [#'sc.lab.example/a #'sc.lab.example/b #'sc.lab.example/x #'sc.lab.example/y #'sc.lab.example/z #'sc.lab.example/u #'sc.lab.example/v]

a 
=> 23
 
x 
=> 3

(+ x z u)
=> 12 

Perhaps surprisingly, defsc is the author's preferred way of recreating the scope. It's usually not as dirty or dangerous as it looks.

If your REPL supports it, you can also achive the same effect by launching a sub-REPL (won't work with nREPL):

(sc.repl/ep-repl 7)

;;;; a, b, u, v etc. will always be in scope from now on

ClojureScript usage

IMPORTANT: In a typical ClojureScript environment, you have to write (sc.api/letsc [7 -3] ...), not (sc.api/letsc 7 ...). Same thing for sc.api/defsc. Writing (sc.api/letsc [7 -3] ...) is also possible in JVM Clojure, just not mandatory.

How come? This is because in ClojureScript, the information linking the Execution Point 7 to the Code Site -3 is not available to the compilation process in charge of macro-expanding letsc and defsc. That's why we have to provide the Code Site ID (-3) explicitly, as a hardcoded number. To achieve a better understanding of this, see the Pitfalls with ClojureScript REPLs Wiki Page.

Project goals

• Providing practical ways of recreating the runtime environment of a piece of code at the REPL
• Targeting a wide range of Clojure execution platforms
• Providing a good foundation for additional tooling, e.g editor integrations
• Being well documented, and friendly to beginners
• Being customizable.

Caveats

Using scope-capture with ClojureScript is supported, but can get tricky: don't forget the above guideline about ClojureScript usage, and when in trouble, see the Pitfalls with ClojureScript REPLs Wiki Page.

Dynamic Vars:

• In order for spy/brk to record dynamic Var bindings, the list of dynamic Vars to observe must be explicitly declared as an option, e.g

(sc.api/spy 
  {:sc/dynamic-vars [*out* my.app/*foo*]}
  expr)

• sc.api/letsc will rebind the dynamic Vars that were captured (using (binding [...] ...)), sc.api/defsc will not.

Local/global name collisions:

• sc.api/defsc will overwrite the global Vars that have the same name as the locals it has captured
• In particular, if a global Var has been defed via defonce, sc.api/defsc won't work for a local of the same name.

For these reasons, using sc.api/letsc or a sub-REPL is generally more error-proof than using defsc, although in many cases it is less practical.

Related tools

Dear tooling authors, if I failed to mention any of your tools, please submit a PR!

Similar

There are other Clojure tools that enable you to recreate the local context in your REPL:

• CIDER: The CIDER Debugger lets you evaluate code in the context of a breakpoint.
• Calva: See Evaluating code in the paused context
• Proto-REPL: See Saving and viewing local bindings values

Complementary

scope-capture has a narrow focus: putting you in the shoes of your program through your REPL. It does not address other debugging concerns, such as visualizing data or observing execution: this can be achieved by complementing this library with other tools, such as tracing libraries, data visualization UIs, pretty-printers, etc. See the Clojure REPL guide for an overview of REPL tools and techniques.

Analogous

Here are tools that achieve a similar effect in other languages:

• Pry (Ruby)
• ipdb (Python)
• the Node.js Debugger offers a repl command to evaluate code at a breakpoint. (JavaScript - Node)

License

Copyright © 2017-2020 Valentin Waeselynck and contributors.

Distributed under the MIT license.