Btrfs source repositories

From btrfs Wiki
(Difference between revisions)
Jump to: navigation, search
(Official repositories: make devel repos more visible)
(Kernel module: update branches)
 
(15 intermediate revisions by 5 users not shown)
Line 11: Line 11:
  
 
Everyone tests with the latest btrfs code from git. Even the latest Linus kernel probably doesn't have the latest, so if you're really interested in using and testing btrfs, get btrfs from git. Distributions usually offer a way to install a kernel built from git.
 
Everyone tests with the latest btrfs code from git. Even the latest Linus kernel probably doesn't have the latest, so if you're really interested in using and testing btrfs, get btrfs from git. Distributions usually offer a way to install a kernel built from git.
 +
 +
= Kernel module =
 +
 +
The kernel.org git repository is not used for development, only for pull requests that go to Linus and linux-next integration:
 +
 +
* https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git -- pull requests, branch ''for-next'' gets pulled to the linux-next tree
 +
 +
The following git repositories are used for development and are updated with
 +
patches from the mailinglist:
 +
 +
* https://github.com/kdave/btrfs-devel
 +
* https://gitlab.com/kdave/btrfs-devel
 +
 +
Branches are usually pushed to both repositories, either can be used.
 +
 +
There are:
 +
 +
* main queue with patches for next development cycle (branch name ''misc-next'')
 +
* queue with patches for current release cycle (the name has the version, eg ''for-4.15'' or ''misc-4.15'').
 +
* topic branches, eg. from a patchset picked from mailinglist
 +
* snapshots of ''for-next'', that contain all of the above (eg. for-next-20200512)
 +
 +
Note that the branches get rebased.  The base point for patches depend on the
 +
development phase.  See [[Developer%27s_FAQ#Development_schedule]].
 +
Independent changes can be based on the ''linus/master'' branch, changes that
 +
could depend on patches that have been added to one of the queues should use
 +
that as a base.
  
 
= btrfs-progs git repository =
 
= btrfs-progs git repository =
Line 18: Line 45:
 
The sources of the userspace utilities can be obtained from these repositories:
 
The sources of the userspace utilities can be obtained from these repositories:
  
* git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git ([http://git.kernel.org/?p=linux/kernel/git/kdave/btrfs-progs.git;a=summary gitweb access])
+
* git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git ([http://git.kernel.org/?p=linux/kernel/git/kdave/btrfs-progs.git;a=summary gitweb access]) -- release repisotiry, not for development
* git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git ([http://git.kernel.org/?p=linux/kernel/git/mason/btrfs-progs.git;a=summary gitweb access])
+
  
There are 2 repositories that get synced at the time of a release ([http://article.gmane.org/gmane.comp.file-systems.btrfs/38063 announcement]) and can be considered equal for non-development branches.
+
The '''master''' branch contains the latest released version and is never rebased.
  
The '''master''' branch of the repositories is updated at the time of a release and is not used for development. This branch is never rebased.
+
Development git repositories:
 
+
Devleopment git repositories:
+
  
 
* git://github.com/kdave/btrfs-progs.git ([https://github.com/kdave/btrfs-progs web access])
 
* git://github.com/kdave/btrfs-progs.git ([https://github.com/kdave/btrfs-progs web access])
* git://repo.or.cz/btrfs-progs-unstable/devel.git ([http://repo.or.cz/w/btrfs-progs-unstable/devel.git web access])
+
* git://gitlab.com/kdave/btrfs-progs.git ([https://gitlab.com/kdave/btrfs-progs web access])
 +
* <strike>git://repo.or.cz/btrfs-progs-unstable/devel.git ([http://repo.or.cz/w/btrfs-progs-unstable/devel.git web access])</strike> -- not used anymore
 +
 
 +
For build dependencies and installation instructions please see [https://github.com/kdave/btrfs-progs/blob/master/INSTALL]
  
 
== Development branches ==
 
== Development branches ==
Line 38: Line 65:
 
== Integration branches ==
 
== Integration branches ==
  
'''Integration branches''' collect patches that may need public testing and are provided for conveniece to developers or users willing to test unstable version. The patches in these branches are not guaranteed to be merged. The branches are assembled from scratch and server more like a snapshots than stable bases for other development.
+
'''Integration branches''' collect patches that may need public testing and are provided for convenience to developers or users willing to test unstable version. The patches in these branches are not guaranteed to be merged. The branches are assembled from scratch and serve more like a snapshots than stable bases for other development.
  
  git://repo.or.cz/btrfs-progs-unstable/devel.git integration-''YYYYMMDD''
+
  git://github.com/kdave//btrfs-progs.git integration-''YYYYMMDD''
  
where ''YYYMMDD'' is the date of the branch.
+
where ''YYYYMMDD'' is the date of the branch.
  
 
To check out this repository, and select the latest branch, use:
 
To check out this repository, and select the latest branch, use:
  
  git clone git://repo.or.cz/btrfs-progs-unstable/devel.git
+
  git clone git://github.com/kdave//btrfs-progs.git
  cd devel
+
  cd btrfs-progs
 
  git checkout integration-''YYYYMMDD''
 
  git checkout integration-''YYYYMMDD''
 
Mirror of the repository is at
 
git://github.com/kdave/btrfs-progs.git
 
 
and has the same layout as the previous repository.
 
  
 
Note that this branch can be rebased when there's good reason for that (adding Reported-by/Reviewed-by tags to the middle of the patch series) or if there's a change needed down the patch stack in general. The point is to keep the git history clean.
 
Note that this branch can be rebased when there's good reason for that (adding Reported-by/Reviewed-by tags to the middle of the patch series) or if there's a change needed down the patch stack in general. The point is to keep the git history clean.
Line 68: Line 90:
 
* '''one logical change''' per patch: eg. not mixing bugfixes, cleanups, features etc., sometimes it's not clear and will be usually pointed out during reviews
 
* '''one logical change''' per patch: eg. not mixing bugfixes, cleanups, features etc., sometimes it's not clear and will be usually pointed out during reviews
  
= btrfs kernel module git repository =
+
= Administration and support tools =
  
Until October 2012, the latest features and bug fixes hit this repo before going upstream to Linus' kernel tree. Currently it is not up to date. The Linux RCs are more up to date and btrfs-next (see below) is most up to date.
+
There is a separate repository of useful scripts for common administrative tasks on btrfs. This is at:
  
btrfs repository (the command downloads a complete Linux kernel tree, if you have a local instance already, add it as a git remote) ([http://git.kernel.org/?p=linux/kernel/git/mason/linux-btrfs.git;a=summary Gitweb access])
+
https://github.com/kdave/btrfsmaintenance/
 
+
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git
+
 
+
== Integration repository (btrfs-next) ==
+
 
+
Josef Bacik maintains an "integration" branch of all the kernel patches  seen on the mailing list.
+
git clone git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git
+
[http://git.kernel.org/?p=linux/kernel/git/josef/btrfs-next.git;a=summary gitweb access]
+
 
+
This is often the code you want to test if you think you've found a bug or want to test the latest features.
+
Check the git log for the fs/btrfs to check which is up-to-date - in case one
+
tree has fallen behind e.g.
+
 
+
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/fs/btrfs
+
 
+
vs.
+
 
+
https://git.kernel.org/cgit/linux/kernel/git/josef/btrfs-next.git/log/fs/btrfs
+
 
+
vs.
+
 
+
https://git.kernel.org/cgit/linux/kernel/git/mason/linux-btrfs.git/log/fs/btrfs (use the drop-down at the far top-right of this window to check the different branches which Chris maintains - the latest code is normally not on 'master').
+
 
+
"Hello,
+
 
+
In an effort to be a little bit better about reviewing patches sent to the
+
mailinglist I've created a btrfs-next git tree kernel.org.  You can get it here
+
 
+
git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git
+
 
+
I'm going to be scraping the mailinglist every day for patches that are sent and
+
applying them to this tree in order to get some better testing throughout the
+
development cycle and possibly stop being such a lazy bum about reviewing.  If
+
you are looking to get patches into the next merge window and you notice I
+
haven't included them in this tree please let me know so I can pull them in, I
+
did a very casual look through the list to find stuff so I probably missed
+
something.  Thanks much,
+
 
+
Josef"
+
  
 
= Patches sent to mailinglist =
 
= Patches sent to mailinglist =
Line 122: Line 105:
  
 
you may want to add ''--reject'', or decide otherwise what to do with the patch.
 
you may want to add ''--reject'', or decide otherwise what to do with the patch.
 
= Build =
 
== Dependencies ==
 
 
Before compiling the userspace tools you'll need to get some dependencies:
 
 
=== Fedora ===
 
 
yum install libuuid-devel libattr-devel zlib-devel libacl-devel e2fsprogs-devel libblkid-devel lzo2-devel asciidoc xmlto
 
 
Note that some Fedora/Red Hat derived distributions use lzo-devel instead of lzo2-devel.
 
 
=== Debian/Ubuntu ===
 
 
sudo apt-get install asciidoc xmlto --no-install-recommends
 
 
Followed by either of:
 
sudo apt-get build-dep btrfs-tools
 
-or-
 
sudo apt-get install uuid-dev libattr1-dev zlib1g-dev libacl1-dev e2fslibs-dev libblkid-dev liblzo2-dev
 
 
If you tried the first and get "cannot find llzo2", or "blkid/blkid.h: No such file or directory" try the second; your distro has an even older version of btrfs-progs packaged than expected.
 
 
=== Arch Linux ===
 
 
pacman -S e2fsprogs
 
 
=== openSUSE ===
 
 
zypper in libattr-devel zlib-devel libacl-devel libext2fs-devel libuuid-devel libblkid-devel lzo-devel asciidoc xmlto
 
 
== Build ==
 
 
Once you have everything in place, you can run
 
 
./autogen.sh
 
./configure
 
make
 
 
to build the user-space tools. After that they can be used like this:
 
 
./btrfs fi show
 

Latest revision as of 21:45, 15 July 2020

Since 2.6.29-rc1, Btrfs has been included in the mainline kernel.

Warning, Btrfs evolves very quickly do not test it unless:

  1. You have good backups and you have tested the restore capability
  2. You have a backup installation that you can switch to when something breaks
  3. You are willing to report any issues you find
  4. You can apply patches and compile the latest btrfs code against your kernel (quite easy with git and dkms, see below)
  5. You acknowledge that btrfs may eat your data
  6. Backups! Backups! Backups!

Everyone tests with the latest btrfs code from git. Even the latest Linus kernel probably doesn't have the latest, so if you're really interested in using and testing btrfs, get btrfs from git. Distributions usually offer a way to install a kernel built from git.

Contents

[edit] Kernel module

The kernel.org git repository is not used for development, only for pull requests that go to Linus and linux-next integration:

The following git repositories are used for development and are updated with patches from the mailinglist:

Branches are usually pushed to both repositories, either can be used.

There are:

  • main queue with patches for next development cycle (branch name misc-next)
  • queue with patches for current release cycle (the name has the version, eg for-4.15 or misc-4.15).
  • topic branches, eg. from a patchset picked from mailinglist
  • snapshots of for-next, that contain all of the above (eg. for-next-20200512)

Note that the branches get rebased. The base point for patches depend on the development phase. See Developer's_FAQ#Development_schedule. Independent changes can be based on the linus/master branch, changes that could depend on patches that have been added to one of the queues should use that as a base.

[edit] btrfs-progs git repository

[edit] Official repositories

The sources of the userspace utilities can be obtained from these repositories:

The master branch contains the latest released version and is never rebased.

Development git repositories:

For build dependencies and installation instructions please see [1]

[edit] Development branches

The latest development branch is called devel. Contains patches that are reviewed or tested and on the way to the next release. When a patch is added to the branch, a mail notification is sent as a reply to the patch.

The git repositories on kernel.org are not used for development or integration branches.

[edit] Integration branches

Integration branches collect patches that may need public testing and are provided for convenience to developers or users willing to test unstable version. The patches in these branches are not guaranteed to be merged. The branches are assembled from scratch and serve more like a snapshots than stable bases for other development.

git://github.com/kdave//btrfs-progs.git integration-YYYYMMDD

where YYYYMMDD is the date of the branch.

To check out this repository, and select the latest branch, use:

git clone git://github.com/kdave//btrfs-progs.git
cd btrfs-progs
git checkout integration-YYYYMMDD

Note that this branch can be rebased when there's good reason for that (adding Reported-by/Reviewed-by tags to the middle of the patch series) or if there's a change needed down the patch stack in general. The point is to keep the git history clean.

[edit] Note to GitHub users

The pull requests will not be accepted directly, the preferred way is to send patches to the mailinglist instead. You can link to a branch in any git repository if the mails do not make it to the mailinglist or for convenience.

The development model of btrfs-progs shares a lot with the kernel model. The github way is different in some ways. We, the upstream community, expect that the patches meet some criteria (often lacking in github contributions):

  • proper subject line: eg. prefix with btrfs-progs: subpart, ... , descriptive yet not too long
  • proper changelog: the changelogs are often missing or lacking explanation why the change was made, or how is something broken, what are user-visible effects of the bug or the fix, how does an improvement help or the intended usecase
  • the Signed-off-by line: this document who authored the change, you can read more about the The Developer's Certificate of Origin here (chapter 11)
  • one logical change per patch: eg. not mixing bugfixes, cleanups, features etc., sometimes it's not clear and will be usually pointed out during reviews

[edit] Administration and support tools

There is a separate repository of useful scripts for common administrative tasks on btrfs. This is at:

https://github.com/kdave/btrfsmaintenance/

[edit] Patches sent to mailinglist

A convenient interface to get an overview of patches and the related mail discussions can be found at https://patchwork.kernel.org/project/linux-btrfs/list/ .

It is possible to directly apply a patch by pasting the mbox link from the patch page to the command

$ wget -O - 'https://patchwork.kernel.org/patch/123456/mbox' | git am -

you may want to add --reject, or decide otherwise what to do with the patch.

Personal tools