Crate turbo_tasks

Source

Expand description

§Turbo Tasks

An incremental computation system that uses macros and types to automate the caching process.

For a high-level overview, start by reading Inside Turbopack: Building Faster by Building Less.

Turbo Tasks defines 4 primitives:

Functions: Units of execution, invalidation, and reexecution.
Values: Data created, stored, and returned by functions.
Traits: Traits that define a set of functions on values.
Collectibles: Values emitted in functions that bubble up the call graph and can be collected in parent functions. Collectibles are deduplicated by cell id equality.

It defines some derived elements from that:

Tasks: An instance of a function together with its arguments.
Vcs (“Value Cells”): References to locations associated with tasks where values are stored. The contents of a cell can change after the reexecution of a function due to invalidation. A Vc can be read to get a read-only reference to the stored data, representing a snapshot of that cell at that point in time.

There are a few design patterns that are commonly used with Turbo Tasks:

Singleton Pattern: Use a private constructor function to ensure a 1:1 mapping between values and value cells.

§Functions and Tasks

#[turbo_tasks::function]s are memoized functions within the build process. An instance of a function with arguments is called a task. They are responsible for:

Tracking Dependencies: Each task keeps track of its dependencies to determine when a recomputation is necessary. Dependencies are tracked when a Vc<T> (Value Cell) is awaited.
Recomputing Changes: When a dependency changes, the affected tasks are automatically recomputed.
Parallel Execution: Every task is spawned as a Tokio task, which uses Tokio’s multithreaded work-stealing executor.

§Task Graph

All tasks and their dependencies form a task graph.

This graph is crucial for invalidation propagation. When a task is invalidated, the changes propagate through the graph, triggering rebuilds where necessary.

§Incremental Builds

Upon execution of functions, turbo-tasks will track which Vcs are read. Once any of these change, turbo-tasks will invalidate the task created from the function’s execution and it will eventually be scheduled and reexecuted.

After initial execution, turbo-tasks employs a bottom-up approach for incremental rebuilds.

By rebuilding invalidated tasks, only the parts of the graph affected by changes are rebuilt, leaving untouched parts intact. No work is done for unchanged parts.

Re-exports§

pub use crate::display::ValueToString;
pub use crate::display::ValueToStringRef;
pub use crate::mapped_read_ref::MappedReadRef;
pub use crate::task::SharedReference;
pub use crate::task::TypedSharedReference;

Modules§

_singleton_pattern: Singleton Pattern
backend
debug
display: Async Formatting
duration_span
event
graph
keyed
mapped_read_ref
message_queue
panic_hooks: Provides a central registry for safe runtime registration and de-registration of panic hooks.
parallel: Parallel for each and map using tokio tasks.
primitives
registry
scope: A scoped tokio spawn implementation that allow a non-’static lifetime for tasks.
small_duration
task
task_statistics
test_helpers
trace
util

Macros§

duration_span: Creates a event-based span that traces a certain duration (lifetime of the guard). It’s not a real span, which means it can be used across threads.
fxindexmap
fxindexset
stringify_path
turbobail: Async bail macro. Resolves arguments then calls anyhow::bail!().
turbofmt: Async format macro. Returns impl Future<Output = Result<RcStr>>.
vdbg: This macro supports the same syntax as dbg!, but also supports pretty-printing Vc types.

Structs§

ApplyEffectsContext
CellId
Completion: Just an empty type, but it’s never equal to itself.
Completions: This is a transparent value type wrapping Vec<ResolvedVc<Completion>>.
CurrentCellRef: A reference to a task’s cell with methods that allow updating the contents of the cell.
Effects: Captured effects from an operation. This struct can be used to return Effects from a turbo-tasks function and apply them later.
EitherTaskInput: A thin wrapper around Either that implements the traits required by TaskInput, notably Encode and Decode.
Error: The Error type, a wrapper around a dynamic error type.
ExecutionId: An identifier for a specific task execution. Used to assert that local Vcs don’t leak. This value may overflow and re-use old values.
InvalidationReasonSet: A set of InvalidationReasons. They are automatically deduplicated and merged by kind during insertion. It implements Display to get a readable representation.
Invalidator: A lightweight handle to invalidate a task. Only stores the task ID. The caller must provide the TurboTasksApi when calling invalidation methods.
JoinHandle
LocalTaskId: Represents the nth local function call inside a task.
OperationVc: A “subtype” (can be converted via .connect()) of Vc that represents a specific call (with arguments) to a task.
PrettyPrintError: Implements Display to print the error message in a friendly way. Puts a summary first and details after that.
ReadCellOptions
ReadOutputOptions
ReadRawVcFuture
ReadRef: The read value of a value cell. The read value is immutable, while the cell itself might change over time. It’s basically a snapshot of a value at a certain point in time.
ReadVcFuture
ResolvedVc: A “subtype” (via Deref) of Vc that represents a specific Vc::cell/.cell() or ResolvedVc::cell/.resolved_cell() constructor call within a task.
SerializationInvalidator
State: This API violates core assumption of turbo-tasks, is believed to be unsound, and there’s no plan fix it. You should prefer to use collectibles instead of state where at all possible. This API may be removed in the future.
TaskId
TraitMethod
TraitRef: Similar to a ReadRef<T>, but contains a value trait object instead.
TraitType
TraitTypeId
TransientInstance: Pass a reference to an instance to a turbo-tasks function.
TransientState
TransientValue: Pass a value by value (Value<Xxx>) instead of by reference (Vc<Xxx>).
TurboTasks
TurboTasksPanic
Unused: A wrapper around a value that is unused.
UpdateInfo
ValueType: A definition of a type of data.
ValueTypeId
Vc: Value cells represent the pending result of a computation, similar to a cell in a spreadsheet. When a Vc’s contents change, the change is propagated by invalidating dependent tasks.
VcCellCompareMode: Mode that compares the cell’s content with the new value and only updates if the new value is different.
VcCellKeyedCompareMode: Mode that compares the cell’s content with the new value key by key and only updates individual keys if the new value is different.
VcCellNewMode: Mode that always updates the cell’s content.
VcDefaultRead: Representation for standard #[turbo_tasks::value], where a read return a reference to the value type[]
VcTransparentRead: Representation for #[turbo_tasks::value(transparent)] types, where reads return a reference to the target type.
VcValueTraitCast: Casts an arbitrary cell content into a TraitRef<T>.
VcValueTypeCast: Casts an arbitrary cell content into a ReadRef<T>.

Enums§

OutputContent: A helper type representing the output of a resolved task.
RawVc: A type-erased representation of Vc.
ReadCellTracking
ReadConsistency
ReadTracking
TaskExecutionReason
TaskPersistence
TaskPriority

Constants§

TRANSIENT_TASK_BIT

Traits§

CollectiblesSource: Implemented on OperationVc and RawVc.
Dynamic: Marker trait that indicates that a Vc<Self> can accept all methods declared on a Vc<T>.
InvalidationReason: A user-facing reason why a task was invalidated. This should only be used for invalidation that were triggered by the user.
InvalidationReasonKind: Invalidation reason kind. This is used to merge multiple reasons of the same kind into a combined description.
JoinIterExt
MagicAny
NonLocalValue: Marker trait indicating that a type does not contain any instances of Vc or references to Vc. It may contain ResolvedVc or OperationVc.
OperationValue: Indicates that a type does not contain any instances of Vc or ResolvedVc. It may contain OperationVc.
OptionVcExt
ShrinkToFit: Recursively calls shrink_to_fit on all elements of the container.
TaskInput: Trait to implement in order for a type to be accepted as a #[turbo_tasks::function] argument.
TryFlatJoinIterExt
TryJoinIterExt
TurboTasksApi: A type-erased subset of TurboTasks stored inside a thread local when we’re in a turbo task context. Returned by the turbo_tasks helper function.
TurboTasksBackendApi: A subset of the TurboTasks API that’s exposed to Backend implementations.
TurboTasksCallApi: Common base trait for TurboTasksApi and TurboTasksBackendApi. Provides APIs for creating tasks from function calls.
Upcast: Marker trait that indicates that a Vc<Self> can be upcasted to a Vc<T>.
UpcastStrict: A specialization of Upcast that ensures that the upcast is strict meaning that T !== Self.
ValueDefault: Vc<T> analog to the Default trait.
VcCast: Trait defined to share behavior between values and traits within ReadRawVcFuture. See VcValueTypeCast and VcValueTraitCast.
VcRead: Trait that controls crate::Vc’s read representation.
VcValueTrait: A trait implemented on all values trait object references that can be used with a value cell (Vc<Box<dyn Trait>>).
VcValueType: A trait implemented on all values types that can be put into a value cell (Vc).

Functions§

apply_effects: Applies all effects that have been emitted by an operations.
block_for_future: Blocking waits for a future to complete. This blocks the current thread potentially staling other concurrent futures (but not other concurrent tasks). Try to avoid this method infavor of awaiting the future instead.
block_in_place: Tells the scheduler about blocking work happening in the current thread. It will make sure to allocate extra threads for the pool.
dynamic_call: Calls TurboTasks::dynamic_call for the current turbo tasks instance.
effect: Schedules an effect to be applied. The passed future is executed once apply_effects is called.
emit
get_effects: Capture effects from an turbo-tasks operation. Since this captures collectibles it might invalidate the current task when effects are changing or even temporary change.
get_invalidator: Get an Invalidator that can be used to invalidate the current task based on external events. Returns None if called outside of a task context.
get_serialization_invalidator: Returns a SerializationInvalidator that can be used to invalidate the serialization of the current task cells.
mark_finished: Marks the current task as finished. This excludes it from waiting for strongly consistency.
mark_session_dependent: Marks the current task as dirty when restored from filesystem cache.
mark_stateful: Marks the current task as stateful. This is used to indicate that the task has interior mutability (e.g., via State or TransientState), which means the task may produce different outputs even with the same inputs.
mark_top_level_task: Marks the current task context as being in a top-level task. When in a top-level task, eventually consistent reads will panic. It is almost always a mistake to perform an eventually consistent read at the top-level of the application.
prevent_gc
run
run_once
run_once_with_reason
spawn: Spawns a future as separate task and returns a JoinHandle which can be used to await the result. The future has access to the current TurboTasks context and runs in the same tracing span. Allocations and cpu time is accounted to the current turbo-tasks function.
spawn_blocking: Spawns a blocking function in a separate task using the blocking pool and returns a JoinHandle which can be used to await the result. The function has access to the current TurboTasks context and runs in the same tracing span. Allocations and cpu time is accounted to the current turbo-tasks function.
spawn_thread: Spawns a thread which runs in background. It has access to the current TurboTasks context, but is not accounted towards the current turbo-tasks function.
trait_call: Calls TurboTasks::trait_call for the current turbo tasks instance.
turbo_tasks
turbo_tasks_scope
turbo_tasks_weak
unmark_top_level_task_may_leak_eventually_consistent_state: Unmarks the current task context as being in a top-level task. The opposite of mark_top_level_task.
with_turbo_tasks

Type Aliases§

FxDashMap
FxIndexMap
FxIndexSet
Result: Result<T, Error>
TaskIdSet

Attribute Macros§

function: Tasks are created by defining a Rust function annotated with the #[turbo_tasks::function] macro and calling it with arguments. Each unique combination of function and arguments create a new task at runtime. Tasks are the fundamental units of work within the build system.
task_storage: Derives the TaskStorage struct and generates optimized storage structures.
value: Implements VcValueType for the given struct or enum. These value types can be used inside of a “value cell” as Vc<...>.
value_impl: A macro used on any impl block for a VcValueType. This can either be an inherent implementation or a trait implementation (see turbo_tasks::value_trait and VcValueTrait).
value_trait: Allows this trait to be used as part of a trait object inside of a value cell, in the form of Vc<Box<dyn MyTrait>>. The annotated trait is made into a subtrait of VcValueTrait.

Derive Macros§

NonLocalValue: Implements NonLocalValue for a struct or enum by adding static (compile-time) assertions that every field implements NonLocalValue.
OperationValue
ShrinkToFit
TaskInput: Refer to the trait documentation for usage.
ValueToString: Derive macro for ValueToString. Also generates ValueToStringify for &T.