libfossil
pages.h
Go to the documentation of this file.
1
/* -*- Mode: C; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
2
/* vim: set ts=2 et sw=2 tw=80: */
3
#if !defined(ORG_FOSSIL_SCM_PAGES_H_INCLUDED)
4
#define ORG_FOSSIL_SCM_PAGES_H_INCLUDED
5
/*
6
Copyright 2013-2021 The Libfossil Authors, see LICENSES/BSD-2-Clause.txt
7
8
SPDX-License-Identifier: BSD-2-Clause-FreeBSD
9
SPDX-FileCopyrightText: 2021 The Libfossil Authors
10
SPDX-FileType: Code
11
12
Heavily indebted to the Fossil SCM project (https://fossil-scm.org).
13
14
*****************************************************************************
15
This file contains only Doxygen-format documentation, split up into
16
Doxygen "pages", each covering some topic at a high level. This is
17
not the place for general code examples - those belong with their
18
APIs.
19
*/
20
21
/** @mainpage libfossil
22
23
Forewarning: this API assumes one is familiar with the Fossil SCM,
24
ideally in detail. The Fossil SCM can be found at:
25
26
https://fossil-scm.org
27
28
libfossil is an experimental/prototype library API for the Fossil
29
SCM. This API concerns itself only with the components of fossil
30
which do not need user interaction or the display of UI components
31
(including HTML and CLI output). It is intended only to model the
32
core internals of fossil, off of which user-level applications
33
could be built.
34
35
The project's repository and additional information can be found at:
36
37
https://fossil.wanderinghorse.net/r/libfossil/
38
39
This code is 100% hypothetical/potential, and does not represent
40
any Official effort of the Fossil project. It is up for any amount
41
of change at any time and does not yet have a stable API.
42
43
All Fossil users are encouraged to participate in its development,
44
but if you are reading this then you probably already knew that
45
:).
46
47
This effort does not represent "Fossil Version 2", but provides an
48
alternate method of accessing and manipulating fossil(1)
49
repositories. Whereas fossil(1) is a monolithic binary, this API
50
provides library-level access to (some level of) the fossil(1)
51
feature set (that level of support grows approximately linearly
52
with each new commit).
53
54
Current status: alpha. Some bits are basically finished but there
55
is a lot of work left to do. The scope is pretty much all
56
Fossil-related functionality which does not require a user
57
interface or direct user interaction, plus some range of utilities
58
to support those which require a UI/user.
59
*/
60
61
/** @page page_terminology Fossil Terminology
62
63
See also: https://fossil-scm.org/home/doc/trunk/www/concepts.wiki
64
65
The libfossil API docs normally assume one is familiar with
66
Fossil-internal terminology, which is of course a silly assumption
67
to make. Indeed, one of libfossil's goals is to make Fossil more
68
accessible, partly be demystifying it. To that end, here is a
69
collection of terms one may come across in the API, along with
70
their meanings in the context of Fossil...
71
72
73
- REPOSITORY (a.k.a. "repo) is an sqlite database file which
74
contains all content for a given "source tree." (We will use the
75
term "source tree" to mean any tree of "source" (documents,
76
whatever) a client has put under Fossil's supervision.)
77
78
- CHECKOUT (a.k.a. "local source tree" or "working copy") refers
79
to (A) the action of pulling a specific version of a repository's
80
state from that repo into the local filesystem, and (B) a local
81
copy "checked out" of a repo. e.g. "he checked out the repo," and
82
"the changes are in his [local] checkout."
83
84
- ARTIFACT is the generic term for anything stored in a repo. More
85
specifically, ARTIFACT refers to "control structures" Fossil uses
86
to internally track changes. These artifacts are stored as blobs
87
in the database, just like any other content. For complete details
88
and examples, see:
89
https://fossil-scm.org/home/doc/tip/www/fileformat.wiki
90
91
- A MANIFEST is a specific type of ARTIFACT - the type which
92
records all metadata for a COMMIT operation (which files, which
93
user, the timestamp, checkin comment, lineage, etc.). For
94
historical reasons, MANIFEST is sometimes used as a generic term
95
for ARTIFACT because what the fossil(1)-internal APIs originally
96
called a Manifest eventually grew into other types of artifacts
97
but kept the Manifest naming convention. In Fossil developer
98
discussion, "manifest" most often means what this page calls
99
ARTIFACT (probably because that how the C code is modelled). The
100
libfossil API calls uses the term "deck" instead of "manifest" to
101
avoid ambiguity/confusion (or to move the confusion somewhere
102
else, at least).
103
104
- CHECKIN is the term libfossil prefers to use for COMMIT
105
MANIFESTS. It is also the action of "checking in"
106
(a.k.a. "committing") file changes to a repository. A CHECKIN
107
ARTIFACT can be one of two types: a BASELINE MANIFEST (or BASELINE
108
CHECKIN) contains a list of all files in that version of the
109
repository, including their file permissions and the UUIDs of
110
their content. A DELTA MANFIEST is a checkin record which derives
111
from a BASELINE MANIFEST and it lists only the file-level changes
112
which happened between the baseline and the delta, recording any
113
changes in content, permisions, or name, and recording
114
deletions. Note that this inheritance of deltas from baselines is
115
an internal optimization which has nothing to do with checkin
116
version inheritance - the baseline of any given delta is normally
117
_not_ its direct checkin version parent.
118
119
- BRANCH, FORK, and TAG are all closely related in Fossil and are
120
explained in detail (with pictures!) at:
121
https://fossil-scm.org/home/doc/trunk/www/concepts.wiki
122
In short: BRANCHes and FORKs are two names for the same thing, and
123
both are just a special-case usage of TAGs.
124
125
- MERGE or MERGING: the process of integrating one version of
126
source code into another version of that source code, using a
127
common parent version as the basis for comparison. This is
128
normally fully automated, but occasionally human (and sometimes
129
Divine) intervention is required to resolve so-called "merge
130
conflicts," where two versions of a file change the same parts of
131
a common parent version.
132
133
- RID (Record ID) is a reference to the blob.rid field in a
134
repository DB. RIDs are used extensively throughout the API for
135
referencing content records, but they are transient values local
136
to a given copy of a given repository at a given point in
137
time. They _can_ change, even for the same content, (e.g. a
138
rebuild can hypothetically change them, though it might not, and
139
re-cloning a repo may very well change some RIDs). Clients must
140
never rely on them for long-term reference to SCM'd data - always use
141
the full UUID of such data. Even though they normally appear to be
142
static, they are most explicitly NOT guaranteed to be. Nor are
143
their values guaranteed to imply any meaning, e.g. "higher is
144
newer" is not necessarily true because synchronization can import
145
new remote content in an arbitrary order and a rebuild might
146
import it in random order. The API uses RIDs basically as handles
147
to arbitrary blob content and, like most C-side handles, must be
148
considered transient in nature. That said, within the db, records
149
are linked to each other exclusively using RIDs, so they do have
150
some persistence guarantees for a given db instance.
151
*/
152
153
154
/** @page page_APIs High-level API Overview
155
156
The primary end goals of this project are to eventually cover the
157
following feature areas:
158
159
- Provide embeddable SCM to local apps using sqlite storage.
160
- Provide a network layer on top of that for synchronization.
161
- Provide apps on top of those to allow administration of repos.
162
163
To those ends, the fossil APIs cover the following categories of
164
features:
165
166
Filesystem:
167
168
- Conversions of strings from OS-native encodings to UTF.
169
fsl_utf8_to_unicode(), fsl_filename_to_utf8(), etc. These are
170
primarily used internally but may also be useful for applications
171
working with files (as most clients will). Actually... most of
172
these bits are only needed for portability across Windows
173
platforms.
174
175
- Locating a user's home directory: fsl_find_home_dir()
176
177
- Normalizing filenames/paths. fsl_file_canonical_name() and friends.
178
179
- Checking for existence, size, and type (file vs directory) with
180
fsl_is_file() and fsl_dir_check(), or the more general-purpose
181
fsl_stat().
182
183
184
Databases (sqlite):
185
186
- Opening/closing sqlite databases and running queries on them,
187
independent of version control features. See fsl_db_open() and
188
friends. The actual sqlite-level DB handle type is abstracted out
189
of the public API, largely to simplify an eventual port from
190
sqlite3 to sqlite4 or (hypothetically) to other storage back-ends
191
(not gonna happen - too much work).
192
193
- There are lots of utility functions for oft-used operations,
194
e.g. fsl_config_get_int32() and friends to fetch settings from one
195
of several different configuration areas (global, repository,
196
checkout, and "versionable" settings).
197
198
- Pseudo-recusive transactions: fsl_db_txn_begin() and
199
fsl_db_txn_end(). sqlite does not support truly nested
200
transactions, but they can be simulated quite effectively so long
201
as certain conventions are adhered to.
202
203
- Cached statements (an optimization for oft-used queries):
204
fsl_db_prepare_cached() and friends.
205
206
207
The DB API is (as Brad Harder put so well) "very present" in the
208
public API. While the core API provides access to the underlying
209
repository data, it cannot begin to cover even a small portion of
210
potential use cases. To that end, it exposes the DB API so that
211
clients who want to custruct their own data can do so. It does
212
require research into the underlying schemas, but gives
213
applications the ability to do _anything_ with their repositories
214
which the core API does not account for. Historically, the ability
215
to create ad-hoc data structures as needed, in the form of SQL
216
queries, has accounted for much of Fossil's feature flexibility.
217
218
219
Deltas:
220
221
- Creation and application of raw deltas, using Fossil's delta
222
format, independent of version control features. See
223
fsl_delta_create() and friends. These are normally used only at
224
the deepest internal levels of fossil, but the APIs are exposed so
225
that clients can, if they wish, use them to deltify their own
226
content independently of fossil's internally-applied
227
deltification. Doing so is remarkably easy, but completely
228
unnecessary for content which will be stored in a repo, as Fossil
229
creates deltas as needed.
230
231
232
SCM:
233
234
- A "context" type (fsl_cx) which manages a repository db and,
235
optionally, a checkout db. Read-only operations on the DB are
236
working and write functionality (adding repo content) is
237
ongoing. See fsl_cx, fsl_cx_init(), and friends.
238
239
- The fsl_deck class assists in parsing, creating, and outputing
240
"artifacts" (manifests, control (tags), events, etc.). It gets its
241
name from it being container for "a collection of cards" (which is
242
what a Fossil artifact is).
243
244
- fsl_content_get() expands a (possibly) deltified blob into its
245
full form, and fsl_content_blob() can be used to fetch a raw blob
246
(possibly a raw delta).
247
248
- A number of routines exist for converting symbol names to RIDs
249
(fsl_sym_to_rid()), UUIDs to RIDs (fsl_uuid_to_rid(),
250
and similar commonly-needed lookups.
251
252
253
Input/Output:
254
255
- The API defines several abstractions for i/o interfaces, e.g.
256
fsl_input_f() and fsl_output_f(), which allow us to accept/emit
257
data from/to arbitrary streamable (as opposed to random-access)
258
sources/destinations. A fsl_cx instance is configured with an
259
output channel, the intention being that all clients of that
260
context should generate any output through that channel, so that
261
all compatible apps can cooperate more easily in terms of i/o. For
262
example, the s2 script binding for libfossil routes fsl_output()
263
through the script engine's i/o channels, so that any output
264
generated by libfossil-using code it links to can take advantage
265
of the script-side output features (such as output buffering,
266
which is needed for any non-trivial CGI output). That said: the
267
library-level code does not actually generate output to that
268
channel, but higher-level code like fcli does, and clients are
269
encouraged to in order to enable their app's output to be
270
redirected to an arbitrary UI element, be it a console or UI
271
widget.
272
273
274
Utilities:
275
276
- fsl_buffer, a generic buffer class, is used heavily by the
277
library. See fsl_buffer and friends.
278
279
- fsl_appendf() provides printf()-like functionality, but sends
280
its output to a callback function (optionally stateful), making it
281
the one-stop-shop for string formatting within the library.
282
283
- The fsl_error class is used to propagate error information
284
between the libraries various levels and the client.
285
286
- The fsl_list class acts as a generic container-of-pointers, and
287
the API provides several convenience routines for managing them,
288
traversing them, and cleaning them up.
289
290
- Hashing: there are a number of routines for calculating SHA1,
291
SHA3, and MD5 hashes. See fsl_sha1_cx, fsl_sha3_cx, fsl_md5_cx,
292
and friends.
293
294
- zlib compression is used for storing artifacts. See
295
fsl_data_is_compressed(), fsl_buffer_compress(), and friends.
296
These are never needed at the client level, but are exposed "just
297
in case" a given client should want them.
298
*/
299
300
/** @page page_is_isnot Fossil is/is not...
301
302
Through porting the main fossil application into library form,
303
the following things have become very clear (or been reinforced)...
304
305
Fossil is...
306
307
- _Exceedingly_ robust. Not only is sqlite literally the single
308
most robust application-agnostic container file format on the
309
planet, but Fossil goes way out of its way to ensure that what
310
gets put in is what gets pulled out. It cuts zero corners on data
311
integrity, even adding in checks which seem superfluous but
312
provide another layer of data integrity (i'm primarily talking
313
about the R-card here, but there are other validation checks). It
314
does this at the cost of memory and performance (that said, it's
315
still easily fast enough for its intended uses). "Robust" doesn't
316
mean that it never crashes nor fails, but that it does so with
317
(insofar as is technically possible) essentially zero chance of
318
data loss/corruption.
319
320
- Long-lived: the underlying data format is independent of its
321
storage format. It is, in principal, usable by systems as yet
322
unconceived by the next generation of programmers. This
323
implementation is based on sqlite, but the model can work with
324
arbitrary underlying storage.
325
326
- Amazingly space-efficient. The size of a repository database
327
necessarily grows as content is modified. However, Fossil's use of
328
zlib-compressed deltas, using a very space-efficient delta format,
329
leads to tremendous compression ratios. As of this writing (March,
330
2021), the main Fossil repo contains approximately 5.36GB of
331
content, were we to check out every single version in its
332
history. Its repository database is only 64MB, however, equating
333
to a 83:1 compression ration. Ratios in the range of 20:1 to 40:1
334
are common, and more active repositories tend to have higher
335
ratios. The TCL core repository, with just over 15 years of code
336
history (imported, of course, as Fossil was introduced in 2007),
337
is (as of September 2013) only 187MB, with 6.2GB of content and a
338
33:1 compression ratio.
339
340
Fossil is not...
341
342
- Memory-light. Even very small uses can easily suck up 1MB of RAM
343
and many operations (verification of the R card, for example) can
344
quickly allocate and free up hundreds of MB because they have to
345
compose various versions of content on their way to a specific
346
version. To be clear, that is total RAM usage, not _peak_ RAM
347
usage. Peak usage is normally a function of the content it works
348
with at a given time, often in direct relation to (but
349
significantly more than) the largest single file processed in a
350
given session. For any given delta application operation, Fossil
351
needs the original content, the new content, and the delta all in
352
memory at once, and may go through several such iterations while
353
resolving deltified content. Verification of its 'R-card' alone
354
can require a thousand or more underlying DB operations and
355
hundreds of delta applications. The internals use caching where it
356
would save us a significant amount of db work relative to the
357
operation in question, but relatively high memory costs are
358
unavoidable. That's not to say we can't optimize a bit, but first
359
make it work, then optimize it. The library takes care to re-use
360
memory buffers where it is feasible (and not too intrusive) to do
361
so, but there is yet more RAM to be optimized away in this regard.
362
*/
363
364
/** @page page_threading Threads and Fossil
365
366
It is strictly illegal to use a given fsl_cx instance from more
367
than one thread. Period.
368
369
It is legal for multiple contexts to be running in multiple
370
threads, but only if those contexts use different
371
repository/checkout databases. Though access to the storage is,
372
through sqlite, protected via a mutex/lock, this library does not
373
have a higher-level mutex to protect multiple contexts from
374
colliding during operations. So... don't do that. One context, one
375
repo/checkout.
376
377
Multiple application instances may each use one fsl_cx instance to
378
share repo/checkout db files, but must be prepared to handle
379
locking-related errors in such cases. e.g. db operations which
380
normally "always work" may suddenly pause for a few seconds before
381
giving up while waiting on a lock when multiple applications use
382
the same database files. sqlite's locking behaviours are
383
documented in great detail at https://sqlite.org.
384
*/
385
386
/** @page page_artifacts Creating Artifacts
387
388
A brief overview of artifact creating using this API. This is targeted
389
at those who are familiar with how artifacts are modelled and generated
390
in fossil(1).
391
392
Primary artifact reference:
393
394
https://fossil-scm.org/home/doc/trunk/www/fileformat.wiki
395
396
In fossil(1), artifacts are generated via the careful crafting of
397
a memory buffer (large string) in the format described in the
398
document above. While it's relatively straightforward to do, there
399
are lots of potential gotchas, and a bug can potentially inject
400
"bad data" into the repo (though the verify-before-commit process
401
will likely catch any problems before the commit is allowed to go
402
through). The libfossil API uses a higher-level (OO) approach,
403
where the user describes a "deck" of cards and then tells the
404
library to save it in the repo (fsl_deck_save()) or output it to
405
some other channel (fsl_deck_output()). The API ensures that the
406
deck's cards get output in the proper order and that any cards
407
which require special treatment get that treatment (e.g. the
408
"fossilize" encoding of certain text fields). The "deck" concept
409
is equivalent to Artifact in fossil(1), but we use the word deck
410
because (A) Artifact is highly ambiguous in this context and (B)
411
deck is arguably the most obvious choice for the name of a type
412
which acts as a "container of cards."
413
414
Ideally, client-level code will never have to create an artifact
415
via the fsl_deck API (because doing so requires a fairly good
416
understanding of what the deck is for in the first place,
417
including the individual Cards). The public API strives to hide
418
those levels of details, where feasible, or at least provide
419
simpler/safer alternatives for basic operations. Some operations
420
may require some level of direct work with a fsl_deck
421
instance. Likewise, much read-only functionality directly exposes
422
fsl_deck to clients, so some familiarity with the type and its
423
APIs will be necessary for most clients.
424
425
The process of creating an artifact looks a lot like the following
426
code example. We have elided error checking for readability
427
purposes, but in fact this code has undefined behaviour if error
428
codes are not checked and appropriately reacted to.
429
430
```
431
fsl_deck deck = fsl_deck_empty;
432
fsl_deck * d = &deck ; // for typing convenience.
433
// Doxygen bug ^^^^^^^ requires space before semicolon!
434
fsl_deck_init( fslCtx, d, FSL_SATYPE_CONTROL ); // must come first
435
fsl_deck_D_set( d, fsl_julian_now() );
436
fsl_deck_U_set( d, "your-fossil-name", -1 );
437
fsl_deck_T_add( d, FSL_TAGTYPE_ADD, "...uuid being tagged...",
438
"tag-name", "optional tag value");
439
...
440
// Now output it to stdout:
441
fsl_deck_output( f, d, fsl_output_f_FILE, stdout );
442
// See also: fsl_deck_save(), which stores it in the db and
443
// "crosslinks" it.
444
fsl_deck_finalize(d);
445
```
446
447
The order the cards are added to the deck is irrelevant - they
448
will be output in the order specified by the Fossil specs
449
regardless of their insertion order. Each setter/adder function
450
knows, based on the deck's type (set via fsl_deck_init()), whether
451
the given card type is legal, and will return an error (probably
452
FSL_RC_TYPE) if an attempt is made to add a card which is illegal
453
for that deck type. Likewise, fsl_deck_output() and
454
fsl_deck_save() confirm that the decks they are given contain (A)
455
only allowed cards and (B) have all required
456
cards. fsl_deck_output() will "unshuffle" the cards, making sure
457
they're in the correct order.
458
459
Sidebar: normally outputing a structure can use a const form of
460
that structure, but the traversal of F-cards in a deck requires
461
(for the sake of delta manifests) using a non-const cursor. Thus
462
outputing a deck requires a non-const instance. If it weren't for
463
delta manifests, we could be "const-correct" here.
464
*/
465
466
/** @page page_transactions DB Transactions
467
468
The fsl_db_txn_begin() and fsl_db_txn_end()
469
functions implement a basic form of recursive transaction,
470
allowing the library to start and end transactions at any level
471
without having to know whether a transaction is already in
472
progress (sqlite3 does not natively support nested
473
transactions). A rollback triggered in a lower-level transaction
474
will propagate the error back through the transaction stack and
475
roll back the whole transaction, providing us with excellent error
476
recovery capabilities (meaning we can always leave the db in a
477
well-defined state).
478
479
It is STRICTLY ILLEGAL to EVER begin a transaction using "BEGIN"
480
or end a transaction by executing "COMMIT" or "ROLLBACK" directly
481
on a fsl_db instance. Doing so bypasses internal state which needs
482
to be kept abreast of things and will cause Grief and Suffering
483
(on the client's part, not mine).
484
485
Tip: implementing a "dry-run" mode for most fossil operations is
486
trivial by starting a transaction before performing the
487
operations. Many operations run in a transaction, but if the
488
client starts one of his own they can "dry-run" any op by simply
489
rolling back the transaction he started. Abstractly, that looks
490
like this pseudocode:
491
492
```
493
db.begin();
494
fsl.something();
495
fsl.somethingElse();
496
if( dryRun ) db.rollback();
497
else db.commit();
498
```
499
*/
500
501
/** @page page_code_conventions Code Conventions
502
503
Project and Code Conventions...
504
505
Foreword: all of this more or less evolved organically or was
506
inherited from fossil(1) (where it evolved organically, or was
507
inherited from sqilte (where it evol...)), and is written up here
508
more or less as a formality. Historically i've not been a fan of
509
coding conventions, but as someone else put it to me, "the code
510
should look like it comes from a single source," and the purpose
511
of this section is to help orient those looking to hack in the
512
sources. Note that most of what is said below becomes obvious
513
within a few minutes of looking at the sources - there's nothing
514
earth-shatteringly new nor terribly controversial here.
515
516
The Rules/Suggestions/Guidelines/etc. are as follows...
517
518
519
- C99 is the basis. It was C89 until 2021-02-12.
520
521
- The canonical build environment uses the most restrictive set of
522
warning/error levels possible. It is highly recommended that
523
non-canonical build environments do the same. Adding -Wall -Werror
524
-pedantic does _not_ guaranty that all C compliance/portability
525
problems can be caught by the compiler, but it goes a long way in
526
helping us to write clean code. The clang compiler is particularly
527
good at catching subtle foo-foo's such as uninitialized variables.
528
529
- API docs (as you may have already noticed), does not (any
530
longer) follow Fossil's comment style, but instead uses
531
Doxygen-friendly formatting. Each comment block MUST start with
532
two or more asterisks, or '*!', or doxygen apparently doesn't
533
understand it
534
(https://www.stack.nl/~dimitri/doxygen/manual/docblocks.html). When
535
adding code snippets and whatnot to docs, please use doxygen
536
conventions if it is not too much of an inconvenience. All public
537
APIs must be documented with a useful amount of detail. If you
538
hate documenting, let me know and i'll document it (it's what i do
539
for fun).
540
541
- Public API members have a fsl_ or FSL_ prefix (fossil_ seems too
542
long). For private/static members, anything goes. Optional or
543
"add-on" APIs (e.g. ::fcli) may use other prefixes, but are
544
encouraged use an "f-word" (as it were), simply out of deference
545
to long-standing software naming conventions.
546
547
- Internal APIs, especially non-static ones, start with `fsl__` or
548
`FSL__`, with two underscores. Such APIs must never be used in
549
client-side code.
550
551
- Public-API structs and functions use lower_underscore_style().
552
Static/internal APIs may use different styles. It's not uncommon
553
to see UpperCamelCase for file-scope structs.
554
555
- Function parameters and function-scope vars have no set
556
conventions - implementors are free to name those however they
557
like.
558
559
- Overall style, especially scope blocks and indentation, should
560
follow Fossil's. We are _not at all_ picky about whether or not
561
there is a space after/before parens in if( foo ), and similar
562
small details, just the overall code pattern and two-space
563
indentation. Hard tabs are verboten.
564
565
- Structs and enums all get the optional typedef so that they do
566
not need to be qualified with 'struct' resp. 'enum' when
567
used. Because of how doxygen tracks those, the typedef should be
568
separate from the struct declaration, rather than combinding
569
those into a single declaration.
570
571
- Function typedefs are named fsl_XXX_f. Implementations of such
572
typedefs/interfaces are typically named fsl_XXX_f_SUFFIX(), where
573
SUFFIX describes the implementation's
574
specialization. e.g. fsl_output_f() is a callback
575
typedef/interface and fsl_output_f_FILE() is a concrete
576
implementation for FILE handles.
577
578
- Enums tend to be named fsl_XXX_e.
579
580
- Functions follow the naming pattern prefix_NOUN_VERB(), rather
581
than the more C-conventional prefix_VERB_NOUN(),
582
e.g. fsl_foo_get() and fsl_foo_set() rather than fsl_get_foo() and
583
fsl_get_foo(). The primary reasons are (A) sortability for
584
document processors and (B) they more naturally match with OO API
585
conventions, e.g. `noun.verb()`. A few cases knowingly violate
586
this convention for the sake of readability or sorting of several
587
related functions (e.g. fsl_db_get_TYPE() instead of
588
fsl_db_TYPE_get()).
589
590
- Structs intended to be creatable on the stack are accompanied by
591
a const instance named fsl_STRUCT_NAME_empty, and possibly by a
592
macro named fsl_STRUCT_NAME_empty_m, both of which are
593
"default-initialized" instances of that struct. This is superiour
594
to using `memset()` for struct initialization because we can
595
define (and document) arbitrary default values and all clients who
596
copy-construct them are unaffected by many types of changes to the
597
struct's signature (though they may need a recompile). The
598
intention of the fsl_STRUCT_NAME_empty_m macro is to provide a
599
struct-embeddable form for use in other structs or
600
copy-initialization of const structs, and the `_m` macro is always
601
used to initialize its const struct counterpart. e.g. the library
602
guarantees that fsl_cx_empty_m (a macro representing an empty
603
fsl_cx instance) holds the same default values as fsl_cx_empty (a
604
const fsl_cx value).
605
606
- Returning int vs fsl_int_t vs fsl_size_t: int is used as a
607
conventional result code. fsl_int_t is often used as a signed
608
length-style result code (e.g. printf() semantics). Unsigned
609
ranges use fsl_size_t. Ints are (also) used as a "triplean" (3
610
potential values, e.g. <0, 0, >0). fsl_int_t also guarantees that
611
it will be 64-bit if available, so can be used for places where
612
large values are needed but a negative value is legal (or handy),
613
e.g. the final arguments for fsl_strndup() and
614
fsl_buffer_append(). The use of the fsl_xxx_t typedefs, rather
615
than (unsigned) int, is primarily for readability/documentation,
616
e.g. so that readers can know immediately that the function uses a
617
given argument or return value following certain API-wide
618
semantics. It also allows us to better define platform-portable
619
printf/scanf-style format modifiers for them (analog to C99's
620
PRIi32 and friends), which often come in handy.
621
622
- Signed vs. unsigned types for size/length arguments: use the
623
fsl_int_t (signed) argument type when the client may legally pass
624
in a negative value as a hint that the API should use fsl_strlen()
625
(or similar) to determine a byte array's length. Use fsl_size_t
626
when no automatic length determination is possible (or desired),
627
to "force" the client to pass the proper length. Internally
628
fsl_int_t is used in some places where fsl_size_t "should" be used
629
because some ported-in logic relies on loop control vars being
630
able to go negative. Additionally, fossil internally uses negative
631
blob lengths to mark phantom blobs, and care must be taken when
632
using fsl_size_t with those.
633
634
- Functions taking elipses (...) are accompanied by a va_list
635
counterpart named the same as the (...) form plus a trailing
636
'v'. e.g. fsl_appendf() and fsl_appendfv(). We do not use the
637
printf()/vprintf() convention because that hoses sorting of the
638
functions in generated/filtered API documentation.
639
640
- Error handling/reporting: please keep in mind that the core code
641
is a library, not an application. The main implication is that
642
all lib-level code needs to check for errors whereever they can
643
happen (e.g. on every single memory allocation, of which there are
644
many) and propagate errors to the caller, to be handled at his
645
discretion. The app-level code (::fcli) is not particularly strict
646
in this regard, and installs its own allocator which abort()s on
647
allocation error, which simplifies app-side code somewhat
648
vis-a-vis lib-level code. When reporting an error can be improved
649
by the inclusion of an error string, functions like
650
fsl_cx_err_set() can be used to report the error. Several of the
651
high-level types in the API have fsl_error object member which
652
contains such error state. The APIs which use that state take care
653
to use-use the error string memory whenever possible, so setting
654
an error string is often a non-allocating operation.
655
*/
656
657
658
/** @page page_fossil_arch Fossil Architecture Overview
659
660
An introduction to the Fossil architecture. These docs
661
are basically just a reformulation of other, more detailed,
662
docs which can be found via the main Fossil site, e.g.:
663
664
- https://fossil-scm.org/home/doc/trunk/www/concepts.wiki
665
666
- https://fossil-scm.org/home/doc/trunk/www/fileformat.wiki
667
668
Fossil's internals are fundamentally broken down into two basic
669
parts. The first is a "collection of blobs." The simplest way to
670
think of this (and it's not far from the full truth) is a
671
directory containing lots of files, each one named after a hash of
672
its contents. This pool contains ALL content required for a
673
repository - all other data can be generated from data contained
674
here. Included in the blob pool are so-called Artifacts. Artifacts
675
are simple text files with a very strict format, which hold
676
information regarding the idententies of, relationships involving,
677
and other metadata for each type of blob in the pool. The most
678
fundamental Artifact type is called a Manifest, and a Manifest
679
tells us, amongst other things, which of the hash-based file names
680
has which "real" file name, which version the parent (or parents!)
681
is (or are), and other data required for a "commit" operation.
682
683
The blob pool and the Manifests are all a Fossil repository really
684
needs in order to function. On top of that basis, other forms of
685
Artifacts provide features such as tagging (which is the basis of
686
branching and merging), wiki pages, and tickets. From those
687
Artifacts, Fossil can create/calculate all sorts of
688
information. For example, as new Artifacts are inserted it
689
transforms the Artifact's metadata into a relational model which
690
sqlite can work with. That leads us to what is conceptually the
691
next-higher-up level, but is in practice a core-most component...
692
693
Storage. Fossil's core model is agnostic about how its blobs are
694
stored, but libfossil and fossil(1) both make heavy use of sqlite
695
to implement many of their features. These include:
696
697
- Transaction-capable storage. It's almost impossible to corrupt a
698
Fossil db in normal use. sqlite3 offers literally the most robust
699
general-purpose file format on the planet.
700
701
- The storage of the raw blobs.
702
703
- Artifact metadata is transformed into various DB structures
704
which allow libfossil to traverse historical data much more
705
efficiently than would be possible without a db-like
706
infrastructure (and everything that implies). These structures are
707
kept up to date as new Artifacts are stored in a repository,
708
either via local edits or synching in remote content. These data
709
are incrementally updated as changes are made to a repo.
710
711
- A tremendous amount of the "leg-work" in processing the
712
repository state is handled by SQL queries, without which the
713
library would easily require 5-10x more code in the form of
714
equivalent hard-coded data structures and corresponding
715
functionality. The db approach allows us to ad-hoc structures as
716
we need them, providing us a great deal of flexibility.
717
718
All content in a Fossil repository is in fact stored in a single
719
database file. Fossil additionally uses another database (a
720
"checkout" db) to keep track of local changes, but the repo
721
contains all "fossilized" content. Each copy of a repo is a
722
full-fledged repo, each capable of acting as a central copy for
723
any number of clones or checkouts.
724
725
That's really all there is to understand about Fossil. How it does
726
its magic, keeping everything aligned properly, merging in
727
content, how it stores content, etc., is all internal details
728
which most clients will not need to know anything about in order
729
to make use of fossil(1). Using libfossil effectively, though,
730
does require learning _some_ amount of how Fossil works. That will
731
require taking some time with _other_ docs, however: see the
732
links at the top of this section for some starting points.
733
734
735
Sidebar:
736
737
- The only file-level permission Fossil tracks is the "executable"
738
(a.k.a. "+x") bit. It internally marks symlinks as a permission
739
attribute, but that is applied much differently than the
740
executable bit and only does anything useful on platforms which
741
support symlinks.
742
743
*/
744
745
#endif
746
/* ORG_FOSSIL_SCM_PAGES_H_INCLUDED */
include
fossil-scm
pages.h
Generated on
for libfossil by
1.17.0