Web lists-archives.com

[GIT PULL] afs: Filesystem driver overhaul




Hi Linus,

Could you pull my kAFS filesystem driver overhaul please?

Note that it is based on net-next/master and tip/timers/core and will need
pulling after both of those.  There's a merge at the base of the branch
that effects this.

The major points of the overhaul are:

 (1) Preliminary groundwork is laid for supporting network-namespacing of
     kAFS.  The remainder of the namespacing work requires some way to pass
     namespace information to submounts triggered by an automount.  This
     requires something like the mount overhaul that's in progress.

 (2) sockaddr_rxrpc is used in preference to in_addr for holding addresses
     internally and add support for talking to the YFS VL server.  With
     this, kAFS can do everything over IPv6 as well as IPv4 if it's talking
     to servers that support it.

 (3) Callback handling is overhauled to be generally passive rather than
     active.  'Callbacks' are promises by the server to tell us about data
     and metadata changes.  Callbacks are now checked when we next touch an
     inode rather than actively going and looking for it where possible.

 (4) File access permit caching is overhauled to store the caching
     information per-inode rather than per-directory, shared over
     subordinate files.  Whilst older AFS servers only allow ACLs on
     directories (shared to the files in that directory), newer AFS servers
     break that restriction.

     To improve memory usage and to make it easier to do mass-key removal,
     permit combinations are cached and shared.

 (5) Cell database management is overhauled to allow lighter locks to be
     used and to make cell records autonomous state machines that look
     after getting their own DNS records and cleaning themselves up, in
     particular preventing races in acquiring and relinquishing the fscache
     token for the cell.

 (6) Volume caching is overhauled.  The afs_vlocation record is got rid of
     to simplify things and the superblock is now keyed on the cell and the
     numeric volume ID only.  The volume record is tied to a superblock and
     normal superblock management is used to mediate the lifetime of the
     volume fscache token.

 (7) File server record caching is overhauled to make server records
     independent of cells and volumes.  A server can be in multiple cells
     (in such a case, the administrator must make sure that the VL services
     for all cells correctly reflect the volumes shared between those
     cells).

     Server records are now indexed using the UUID of the server rather
     than the address since a server can have multiple addresses.

 (8) File server rotation is overhauled to handle VMOVED, VBUSY (and
     similar), VOFFLINE and VNOVOL indications and to handle rotation both
     of servers and addresses of those servers.  The rotation will also
     wait and retry if the server says it is busy.

 (9) Data writeback is overhauled.  Each inode no longer stores a list of
     modified sections tagged with the key that authorised it in favour of
     noting the modified region of a page in page->private and storing a
     list of keys that made modifications in the inode.

     This simplifies things and allows other keys to be used to actually
     write to the server if a key that made a modification becomes useless.

(10) Writable mmap() is implemented.  This allows a kernel to be build
     entirely on AFS.

Note that Pre AFS-3.4 servers are no longer supported, though this can be
added back if necessary (AFS-3.4 was released in 1998).

David
---
The following changes since commit 81445e63e67a1e98b1c2575fa2b406d4289d2754:

  Merge remote-tracking branch 'tip/timers/core' into afs-next (2017-11-13 15:36:33 +0000)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-next-20171113

for you to fetch changes up to 98bf40cd99fcfed0705812b6cbdbb3b441a42970:

  afs: Protect call->state changes against signals (2017-11-13 15:38:21 +0000)

----------------------------------------------------------------
AFS development

----------------------------------------------------------------
David Howells (34):
      Pass mode to wait_on_atomic_t() action funcs and provide default actions
      afs: Lay the groundwork for supporting network namespaces
      afs: Close the rxrpc socket only after purging the servers
      afs: Fix server reaping
      afs: Note the cell in the superblock info also
      afs: Push the net ns pointer to more places
      afs: Add some protocol defs
      afs: Update the cache index structure
      afs: Keep and pass sockaddr_rxrpc addresses rather than in_addr
      afs: Allow IPv6 address specification of VL servers
      afs: Consolidate abort_to_error translators
      afs: Condense afs_call's reply{,2,3,4} into an array
      afs: Potentially return call->reply[0] from afs_make_call()
      afs: Connect up the CB.ProbeUuid
      afs: Fix the afs_uuid struct to make the char-sized fields signed
      afs: Rename struct afs_call server member to cm_server
      afs: Overhaul the callback handling
      afs: Overhaul permit caching
      afs: Overhaul cell database management
      afs: Add an address list concept
      afs: Move server rotation code into its own file
      afs: Overhaul volume and server record caching and fileserver rotation
      afs: Make use of the YFS service upgrade to fully support IPv6
      afs: Only progress call state at end of Tx phase from rxrpc callback
      afs: Fix total-length calculation for multiple-page send
      afs: Fix documentation on # vs % prefix in mount source specification
      afs: Trace the initiation and completion of client calls
      afs: Trace the sending of pages
      afs: Fix directory read/modify race
      afs: Introduce a file-private data record
      afs: Get rid of the afs_writeback record
      afs: Implement shared-writeable mmap
      afs: Trace page dirty/clean
      afs: Protect call->state changes against signals

Marc Dionne (1):
      afs: Use a dynamic port if 7001 is in use

 Documentation/filesystems/afs.txt                  |    4 +-
 arch/mips/kernel/traps.c                           |   14 +-
 drivers/gpu/drm/drm_dp_aux_dev.c                   |    8 +-
 drivers/gpu/drm/i915/selftests/intel_breadcrumbs.c |   10 +-
 drivers/media/platform/qcom/venus/hfi.c            |    8 +-
 fs/afs/Makefile                                    |    5 +-
 fs/afs/addr_list.c                                 |  381 ++++++++
 fs/afs/afs.h                                       |   35 +-
 fs/afs/afs_fs.h                                    |    6 +
 fs/afs/afs_vl.h                                    |   73 +-
 fs/afs/cache.c                                     |  239 +----
 fs/afs/callback.c                                  |  504 +++-------
 fs/afs/cell.c                                      |  887 +++++++++++------
 fs/afs/cmservice.c                                 |   77 +-
 fs/afs/dir.c                                       |  461 +++++----
 fs/afs/file.c                                      |  194 +++-
 fs/afs/flock.c                                     |  159 ++-
 fs/afs/fsclient.c                                  |  830 +++++++++-------
 fs/afs/inode.c                                     |  177 ++--
 fs/afs/internal.h                                  |  905 ++++++++++-------
 fs/afs/main.c                                      |  145 ++-
 fs/afs/misc.c                                      |   38 +-
 fs/afs/proc.c                                      |  247 +++--
 fs/afs/rotate.c                                    |  715 ++++++++++++++
 fs/afs/rxrpc.c                                     |  335 ++++---
 fs/afs/security.c                                  |  378 +++++---
 fs/afs/server.c                                    |  782 +++++++++++----
 fs/afs/server_list.c                               |  153 +++
 fs/afs/super.c                                     |  190 ++--
 fs/afs/vlclient.c                                  |  682 ++++++++++---
 fs/afs/vlocation.c                                 |  720 --------------
 fs/afs/vnode.c                                     | 1025 --------------------
 fs/afs/volume.c                                    |  611 ++++++------
 fs/afs/write.c                                     |  709 ++++++++------
 fs/afs/xattr.c                                     |    4 +-
 fs/btrfs/extent-tree.c                             |   27 +-
 fs/fscache/cookie.c                                |    2 +-
 fs/fscache/internal.h                              |    2 -
 fs/fscache/main.c                                  |    9 -
 fs/nfs/inode.c                                     |    4 +-
 fs/nfs/internal.h                                  |    2 +-
 fs/ocfs2/filecheck.c                               |    8 +-
 include/linux/wait_bit.h                           |   15 +-
 include/trace/events/afs.h                         |  293 ++++++
 include/uapi/linux/magic.h                         |    1 +
 kernel/sched/wait_bit.c                            |   18 +-
 mm/filemap.c                                       |    1 +
 47 files changed, 6769 insertions(+), 5324 deletions(-)
 create mode 100644 fs/afs/addr_list.c
 create mode 100644 fs/afs/rotate.c
 create mode 100644 fs/afs/server_list.c
 delete mode 100644 fs/afs/vlocation.c
 delete mode 100644 fs/afs/vnode.c