diff options
author | Willem Jan Withagen <wjw@digiware.nl> | 2016-10-12 11:11:39 +0200 |
---|---|---|
committer | Willem Jan Withagen <wjw@digiware.nl> | 2016-10-12 18:07:43 +0200 |
commit | 7cd32bc3284fc787b0ba416ed42490399f083cc0 (patch) | |
tree | 95052a34750be915b7cd794c440b4c2988bbf7d0 /README.FreeBSD | |
parent | Merge pull request #11427 from ifed01/wip-bluestore-fix-test (diff) | |
download | ceph-7cd32bc3284fc787b0ba416ed42490399f083cc0.tar.xz ceph-7cd32bc3284fc787b0ba416ed42490399f083cc0.zip |
README.FreeBSD: update to match the bimonthly FreeBSD status report
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
Diffstat (limited to 'README.FreeBSD')
-rw-r--r-- | README.FreeBSD | 146 |
1 files changed, 103 insertions, 43 deletions
diff --git a/README.FreeBSD b/README.FreeBSD index f6762aab687..a4a32c1f5db 100644 --- a/README.FreeBSD +++ b/README.FreeBSD @@ -1,67 +1,127 @@ - The FreeBSD build will build most of the tools in Ceph. -Note that the RBD dependant items will not work since FreeBSD does not have RBS +Note that the RBD dependant items will not work since FreeBSD does not +have RBD (yet) + +I started looking into Ceph, because the HAST solution with CARP and +ggate did not really do what I was looking for. But I'm aiming for +running a Ceph storage cluster of storage nodes that are running ZFS. +End station would be running bhyve on RBD disk that are stored in Ceph. + +Progress from last report: +========================== + +Most important change: + - All test (but one) run to completion for the current selection of + tools. This is only the case for "My Fork" repository. Some of the + commits need to be pulled into the HEAD + + - Cmake is now the only way of building Ceph + - Reworked threading/polling code for the simple socket code. + Now uses a selfpipe, instead of using an odd shutdown() signaling + Linux feature. + - Modified the EventKqueue code to work around the "feature" that + starting threads destroys the the kqueue handles. Just finshed the + code so it is not yet submitted to the main repository Build Prerequisites =================== -Compiling and building Ceph is tested on 11-CURRENT -It uses the CLANG toolset that is available, which needs to be atleast 3.7 -Clang 3.4 (on 10.2-STABLE) does not have all required capabilites to compile everything + Compiling and building Ceph is tested on 12-CURRENT, but I guess + 11-RELEASE will also work. And Clang is at 3.8.0. + It uses the CLANG toolset that is available, 3.7 is not tested, but was + working when that was with 11-CURRENT. + Clang 3.4 (on 10.2-STABLE) does not have all required capabilites to + compile everything The following setup will get things running for FreeBSD: - - Install bash and link it in /bin + - Install bash and link it in /bin # requires root privileges sudo pkg install bash sudo ln -s /usr/local/bin/bash /bin/bash + - Need to add one compatability line to + /usr/include/errno.h + #define ENODATA 87 /* Attribute not found */ + (Otherwise some cython compiles will fail.) + +Getting the FreeBSD work on Ceph: +================================= + + - cd "place to work on this" + git clone https://github.com/wjwithagen/ceph.git + cd ceph + git checkout wip-wjw-freebsd-cmake Building Ceph ============= - - Go and start building using Cmake - ./do_freebsd-cmake.sh - - Go and start building using Autobuild (soon depricated) + - Go and start building ./do_freebsd.sh - + +Note that the old way of building using automake is no longer used. + Parts not (yet) included: ========================= - - RBD kernel client - Rados Block Devices is implemented in the Linux kernel - It seems that there used to be a userspace implementation first. - And perhaps ggated could be used as a template since it does some of the - same, other than just between 2 disks. And it has a userspace counterpart. - * @trociny suggests: - Userspace RBD is available under FreeBSD (both librbd and rbd tool - (without linux specific commands) are buildable). - - BlueStore. - FreeBSD and Linux have different AIO API, and that needs to be made compatible - Next to that is there discussion in FreeBSD about aio_cancel not working for all - devicetypes - - CephFS - Cython tries to access an internal field in dirent which does not compile - -Tests that verify the correct working of the above are also excluded from the testset - -Test not (yet) include: + - RBD + Rados Block Devices is implemented in the Linux kernel + It seems that there used to be a userspace implementation first. + And perhaps ggated could be used as a template since it does some of + the same, other than just between 2 disks. And it has a userspace + counterpart. + - BlueStore. + FreeBSD and Linux have different AIO API, and that needs to be made + compatible Next to that is there discussion in FreeBSD about + aio_cancel not working for all devicetypes + - CephFS + Cython tries to access an internal field in dirent which does not + compile + +Tests that verify the correct working of the above are also excluded +from the testset + +Tests not (yet) include: ======================= - - ceph-detect-init/run-tox.sh - Because the current implementation does not know anything about FreeBSD rc-init. - - Tests that make use on nosestests. - Calling these doest not really work since nostests is not in /usr/bin, and calling - thru /usr/bin/env/nosetests does not work on FreeBSD. - test/pybind/test_ceph_argparse.py - test/pybind/test_ceph_daemon.py + - run-tox-ceph-disk + This is the main test that tests the basic Ceph disk install tool. + One of the next things to complete porting for. + - ceph-detect-init/run-tox.sh + Because the current implementation does not know anything about + FreeBSD rc-init. + - Tests that make use of nosetests. + Calling these does not really work since nosetests is not in + /usr/bin, and /usr/bin/env nosetests does not work on FreeBSD. + test/pybind/test_ceph_argparse.py + test/pybind/test_ceph_daemon.py -Things to investigate: -====================== +Task to do: +=========== + - Build an automated test platform that will build ceph/master on + FreeBSD and report the results back to the Ceph developers. This will + increase the maintainability of the FreeBSD side of things. + Developers are signalled that they are using Linux-isms that will not + compile/run on FreeBSD Ceph has several projects for this: Jenkins, + teuthology, pulpito, ... + But even just a while { compile } loop and report the build data on a + static webpage would do for starters. - - ceph-{osd,mon} need 2 signals before they actually terminate. + - Run integration tests to see if the FreeBSD daemons will work with a + Linux Ceph platform. -Things to improve upon: -======================= + - Get the python tests that are currently excluded to work, and test OKE + + - Compile and test the user space RBD (Rados Block Device). + + - Investigate and see if an in-kernel RBD device could be developed a la + 'ggate' + + - Investigate the keystore, which could be kernel embedded on Linux an + currently prevents building Cephfs and some other parts. + + - Scheduler information is not used atm, because the schedulers work + rather different. But at a certain point in time, this would need some + attention: + in: ./src/common/Thread.cc - - Scheduler information is not used atm, because the schedulers work rather different. - But a a certain point in time, this would need some attention: - in: ./src/common/Thread.cc + - Integrate the FreeBSD /etc/rc.d init scripts in the Ceph stack. Both + for testing, but mainly for running Ceph on production machines. |