Initial import of the muenblock Linux kernel module
authorDennis Wassenberg <dennis.wassenberg@secunet.com>
Wed, 4 Mar 2015 16:14:11 +0000 (17:14 +0100)
committerReto Buerki <reet@codelabs.ch>
Mon, 28 Jan 2019 14:58:48 +0000 (15:58 +0100)
The muenblock Linux kernel module implements a block device driver which
transports block I/O requests (bio) via shared memory channels
provided by the Muen Separation Kernel [1].

The server part enables the export of complete block devices or single
partitions from a Linux subject to the driver's client part running on
another Linux subject.

See the README.md file for more information.

[1] - https://muen.sk

.gitignore [new file with mode: 0644]
COPYING [new file with mode: 0644]
Makefile [new file with mode: 0644]
README.md [new file with mode: 0644]
TODO [new file with mode: 0644]
client.c [new file with mode: 0644]
common.c [new file with mode: 0644]
common.h [new file with mode: 0644]
log.h [new file with mode: 0644]
server.c [new file with mode: 0644]

diff --git a/.gitignore b/.gitignore
new file mode 100644 (file)
index 0000000..eabc207
--- /dev/null
@@ -0,0 +1,8 @@
+*.o
+*~
+*.ko
+Module.*
+*.cmd
+.tmp_versions
+modules.order
+*.mod.c
diff --git a/COPYING b/COPYING
new file mode 100644 (file)
index 0000000..d159169
--- /dev/null
+++ b/COPYING
@@ -0,0 +1,339 @@
+                    GNU GENERAL PUBLIC LICENSE
+                       Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
+ 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The licenses for most software are designed to take away your
+freedom to share and change it.  By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users.  This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it.  (Some other Free Software Foundation software is covered by
+the GNU Lesser General Public License instead.)  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+  To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have.  You must make sure that they, too, receive or can get the
+source code.  And you must show them these terms so they know their
+rights.
+
+  We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+  Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software.  If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+  Finally, any free program is threatened constantly by software
+patents.  We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary.  To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                    GNU GENERAL PUBLIC LICENSE
+   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+  0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License.  The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language.  (Hereinafter, translation is included without limitation in
+the term "modification".)  Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope.  The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+  1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+  2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+    a) You must cause the modified files to carry prominent notices
+    stating that you changed the files and the date of any change.
+
+    b) You must cause any work that you distribute or publish, that in
+    whole or in part contains or is derived from the Program or any
+    part thereof, to be licensed as a whole at no charge to all third
+    parties under the terms of this License.
+
+    c) If the modified program normally reads commands interactively
+    when run, you must cause it, when started running for such
+    interactive use in the most ordinary way, to print or display an
+    announcement including an appropriate copyright notice and a
+    notice that there is no warranty (or else, saying that you provide
+    a warranty) and that users may redistribute the program under
+    these conditions, and telling the user how to view a copy of this
+    License.  (Exception: if the Program itself is interactive but
+    does not normally print such an announcement, your work based on
+    the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole.  If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works.  But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+  3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+    a) Accompany it with the complete corresponding machine-readable
+    source code, which must be distributed under the terms of Sections
+    1 and 2 above on a medium customarily used for software interchange; or,
+
+    b) Accompany it with a written offer, valid for at least three
+    years, to give any third party, for a charge no more than your
+    cost of physically performing source distribution, a complete
+    machine-readable copy of the corresponding source code, to be
+    distributed under the terms of Sections 1 and 2 above on a medium
+    customarily used for software interchange; or,
+
+    c) Accompany it with the information you received as to the offer
+    to distribute corresponding source code.  (This alternative is
+    allowed only for noncommercial distribution and only if you
+    received the program in object code or executable form with such
+    an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it.  For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable.  However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+  4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License.  Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+  5. You are not required to accept this License, since you have not
+signed it.  However, nothing else grants you permission to modify or
+distribute the Program or its derivative works.  These actions are
+prohibited by law if you do not accept this License.  Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+  6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions.  You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+  7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all.  For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices.  Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+  8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded.  In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+  9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number.  If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation.  If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+  10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission.  For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this.  Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+                            NO WARRANTY
+
+  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software; you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 2 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License along
+    with this program; if not, write to the Free Software Foundation, Inc.,
+    51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+    Gnomovision version 69, Copyright (C) year name of author
+    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary.  Here is a sample; alter the names:
+
+  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+  `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+  <signature of Ty Coon>, 1 April 1989
+  Ty Coon, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs.  If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.
diff --git a/Makefile b/Makefile
new file mode 100644 (file)
index 0000000..eb628b9
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,23 @@
+CLIENT_MOD   = muenblock-client
+SERVER_MOD   = muenblock-server
+MODULE_FILES = $(SERVER_MOD).ko $(CLIENT_MOD).ko
+
+obj-m = $(MODULE_FILES:.ko=.o)
+
+$(CLIENT_MOD)-objs = common.o client.o
+$(SERVER_MOD)-objs = common.o server.o
+
+KERNEL_SOURCE ?= /lib/modules/$(shell uname -r)/build
+
+PWD := $(shell pwd)
+
+all: compile-module
+compile-module:
+       $(MAKE) -C $(KERNEL_SOURCE) M=$(PWD) modules
+
+install: compile-module
+       install -d $(DESTDIR)/lib/modules/extra
+       install -m 644 $(MODULE_FILES) $(DESTDIR)/lib/modules/extra
+
+clean:
+       $(MAKE) -C $(KERNEL_SOURCE) M=$(PWD) clean
diff --git a/README.md b/README.md
new file mode 100644 (file)
index 0000000..70f8f16
--- /dev/null
+++ b/README.md
@@ -0,0 +1,83 @@
+# Muenblock
+
+## Introduction
+
+The *muenblock* Linux kernel module implements a block device driver which
+transports block I/O requests (bio) via shared memory channels provided by the
+[Muen Separation Kernel][1].
+
+The server part enables the export of entire block devices or single
+partitions from a Linux subject to the driver's client part running on another
+Linux subject.
+
+Note: The *muenblock* kernel modules are in an experimental state and not yet
+ready for production use.
+
+## Usage
+
+The following command inserts the *muenblock-server* module into the kernel on
+the server side. It exports the complete `sda` disk via the *blockdev_request1*
+and *blockdev_response1* channels to the client module. The reader and writer
+protocols are arbitrary values which must match between communicating
+endpoints.
+
+    $ modprobe muenblock-server \
+               req_shm_name=blockdev_request1 \
+               resp_shm_name=blockdev_response1 \
+               req_shm_protocol=9570208dca77db19 \
+               resp_shm_protocol=9851be3282fef0dc \
+               device_name=/dev/sda
+
+The following command is then executed in the client virtual machine to insert
+the *muenblock-client* module:
+
+       $ modprobe muenblock-client \
+               req_shm_name=blockdev_request1 \
+               resp_shm_name=blockdev_response1 \
+               req_shm_protocol=9570208dca77db19 \
+               resp_shm_protocol=9851be3282fef0dc
+
+This leads to the creation of a */dev/muendiska* block device on the client
+which can be used like a directly assigned disk (see the following sections
+regarding supported filesystems and limitations).
+
+It is also possible to only export specific partitions of a block device, see
+the following two example commands on the server and client respectively:
+
+       $ depmod muenblock-server \
+               req_shm_name=blockdev_request1,blockdev_request2 \
+               resp_shm_name=blockdev_response1,blockdev_response2 \
+               req_shm_protocol=9570208dca77db19,9570208dca77db20 \
+               resp_shm_protocol=9851be3282fef0dc,9851be3282fef0dd \
+               device_name=/dev/sda1,/dev/sda5
+
+       $ depmod muenblock-client \
+               req_shm_name=blockdev_request1,blockdev_request2 \
+               resp_shm_name=blockdev_response1,blockdev_response2 \
+               req_shm_protocol=9570208dca77db19,9570208dca77db20 \
+               resp_shm_protocol=9851be3282fef0dc,9851be3282fef0dd
+
+Each partition will appear as separate *muendiska* and *muendiskb* devices in
+the client.
+
+## I/O Scheduler Performance
+
+To increase performance, set the I/O scheduler of the block device exported by
+the *muenblock* server module to `noop` (or `none`, depending on the kernel
+version):
+
+ # echo noop > /sys/class/block/sda/queue/scheduler
+
+## Supported Filesystems
+
+Filesystems of the ext\* family have been extensively tested, also in layered
+modes (ext filesystem on dm-crypt, LVM or both) and are known to work. Other
+filesystems are not currently supported and might not work. *Use at your own
+risk.*
+
+## Limitations
+
+- Graceful server restart is not yet implemented, re-insert the client module if
+the server part vanishes.
+
+[1]: https://muen.sk
diff --git a/TODO b/TODO
new file mode 100644 (file)
index 0000000..0c66b34
--- /dev/null
+++ b/TODO
@@ -0,0 +1,3 @@
+- Adjust blkdev name (see Linux commit c0aa3e0)
+- Extend/improve module param checking
+- Make mods IRQ driven
diff --git a/client.c b/client.c
new file mode 100644 (file)
index 0000000..07726eb
--- /dev/null
+++ b/client.c
@@ -0,0 +1,970 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * Muen block I/O device driver.
+ *
+ * Copyright (C) 2018  secunet Security Networks AG
+ *
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/init.h>
+#include <linux/hdreg.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/major.h>
+#include <linux/blkdev.h>
+#include <linux/bio.h>
+#include <linux/kernel.h>
+#include <linux/kthread.h>
+#include <linux/fs.h>
+#include <linux/cdrom.h>
+
+#include "common.h"
+#include "log.h"
+
+#define UNORDERED_RESPONSES 1
+
+static char *req_shm_name[MUENBLK_MINOR_COUNT];
+static char *resp_shm_name[MUENBLK_MINOR_COUNT];
+
+static char *req_shm_protocol[MUENBLK_MINOR_COUNT];
+static char *resp_shm_protocol[MUENBLK_MINOR_COUNT];
+
+static int dev_name_count;
+
+static unsigned int poll_interval = 1;
+
+static struct muenblk_device *blk_dev[MUENBLK_MINOR_COUNT];
+
+static void muenblk_client_send_server(struct muenblk_device *dev,
+       struct bio *bio, struct muenblk_event *event)
+{
+       unsigned long flags = 0;
+#ifdef UNORDERED_RESPONSES
+       event->priv = (uint64_t) bio;
+#endif
+
+       /*
+        * Flow control: Wait until in-flight requests are below channel
+        * capacity channel_queue_size.
+        */
+       if (atomic_read(&dev->resp.in_flight) >= dev->req.channel_queue_size)
+               wait_event_interruptible
+                       (dev->flow_wq,
+                        atomic_read(&dev->resp.in_flight) <
+                        dev->req.channel_queue_size);
+
+       TRACE("Adding bio to response bio queue" BIO_TRACE(bio));
+
+       spin_lock_irqsave(&dev->resp.lock, flags);
+
+       bio_list_add(&dev->resp.bio_list, bio);
+       atomic_inc(&dev->resp.in_flight);
+
+       spin_unlock_irqrestore(&dev->resp.lock, flags);
+
+       TRACE("Sending request event to server" EVENT_TRACE(event));
+       muen_channel_write(dev->req.channel_data, event);
+
+       wake_up_interruptible(&dev->resp.wq);
+}
+
+static int muenblk_client_send_command_req(struct muenblk_device *dev,
+       struct bio *bio, void *data, size_t size, enum muenblk_command command)
+{
+       int error = 0;
+
+       dev->req.event->type = MUENBLK_EVENT_COMMAND;
+       dev->req.event->id = command;
+       dev->req.event->error = 0;
+       dev->req.event->priv = 0;
+
+       if (data && size) {
+               if (unlikely(size > EVENT_BLOCK_SIZE)) {
+                       ERROR("Failed to send command to server because data exceeds buffer"
+                                       EVENT_TRACE(dev->req.event));
+                       error = -EINVAL;
+                       goto finish;
+               }
+
+               TRACE("Copying command request data to event buffer"
+                               EVENT_TRACE(dev->req.event));
+               memcpy(dev->req.event->data, data, size);
+       }
+
+       muenblk_client_send_server(dev, bio, dev->req.event);
+
+finish:
+       return error;
+}
+
+static int muenblk_client_command_resp(struct muenblk_device *dev,
+       struct bio *bio, struct muenblk_event *event)
+{
+       int error = 0;
+
+       TRACE("Processing COMMAND response" EVENT_TRACE(event));
+
+       if (unlikely(event->error)) {
+               error = event->error;
+
+               ERROR("Command response include error(s) %d\n",
+                       event->error);
+       }
+
+       switch (event->id) {
+       case MUENBLK_COMMAND_SIZE:
+               spin_lock(&dev->wq.lock);
+
+               bio->bi_vcnt = error;
+               bio->bi_opf = REQ_OP_WRITE;
+
+               memcpy(&bio->bi_iter.bi_sector, event->data,
+                       sizeof(bio->bi_iter.bi_sector));
+
+               TRACE("Copied command response data to dummy bio"
+                       BIO_TRACE(bio));
+
+               wake_up_locked(&dev->wq);
+
+               spin_unlock(&dev->wq.lock);
+
+               break;
+
+       default:
+               // nothing to do!
+               break;
+       }
+
+       return error;
+}
+
+static ssize_t muenblk_client_write_segment(struct muenblk_device *dev,
+       struct bio_vec *bvec, char *buf)
+{
+       char *page_ptr = NULL;
+       char *data_ptr = NULL;
+       uint32_t page_offset = 0;
+       int32_t size = 0;
+
+       page_ptr = kmap_atomic(bvec->bv_page);
+       page_offset = bvec->bv_offset;
+       data_ptr = page_ptr + page_offset;
+       size = bvec->bv_len;
+
+       TRACE("Writing segment data from page to server request event"
+               COPY_TRACE(buf, data_ptr, size));
+
+       memcpy(buf, data_ptr, size);
+
+//     print_hex_dump(KERN_INFO, "client event", DUMP_PREFIX_OFFSET, 16, 1,
+//             buf, EVENT_BLOCK_SIZE, true);
+//     print_hex_dump(KERN_INFO, "client page", DUMP_PREFIX_OFFSET, 16, 1,
+//             data_ptr, EVENT_BLOCK_SIZE, true);
+
+       kunmap_atomic(page_ptr);
+
+       return size;
+}
+
+static int muenblk_client_write_req(struct muenblk_device *dev, struct bio *bio)
+{
+       ssize_t size;
+       struct bio_vec bvec;
+       sector_t sector = bio->bi_iter.bi_sector >> 3;
+
+       dev->req.event->type = MUENBLK_EVENT_WRITE;
+       dev->req.event->id = sector;
+
+       bvec = bio_iovec(bio);
+       size = muenblk_client_write_segment(dev, &bvec,
+                                           dev->req.event->data);
+
+       if (size < 0)
+               return size;
+
+       BUG_ON(size != bvec.bv_len);
+
+       muenblk_client_send_server(dev, bio, dev->req.event);
+
+       return 0;
+}
+
+static ssize_t muenblk_client_read_segment(struct muenblk_device *dev,
+       struct bio_vec *bvec, char *buf)
+{
+       char *page_ptr = NULL;
+       char *data_ptr = NULL;
+       char *buf_ptr = NULL;
+       uint32_t page_offset = 0;
+       int32_t size = 0;
+
+       page_ptr = kmap_atomic(bvec->bv_page);
+       page_offset = bvec->bv_offset;
+       data_ptr = page_ptr + page_offset;
+       size = bvec->bv_len;
+       buf_ptr = buf + page_offset;
+
+       TRACE("Reading: page_offset %u", page_offset);
+       TRACE("Reading segment data from server response event to page"
+               COPY_TRACE(data_ptr, buf_ptr, size));
+
+       memcpy(data_ptr, buf_ptr, size);
+
+//     print_hex_dump(KERN_INFO, "client", DUMP_PREFIX_OFFSET, 16, 1,
+//             buf_ptr, EVENT_BLOCK_SIZE, true);
+//     print_hex_dump(KERN_INFO, "client page", DUMP_PREFIX_OFFSET, 16, 1,
+//             data_ptr, EVENT_BLOCK_SIZE, true);
+
+       kunmap_atomic(page_ptr);
+
+       return size;
+}
+
+static int muenblk_client_write_resp(struct muenblk_device *dev,
+       struct bio *bio, struct muenblk_event *event)
+{
+       int error = 0;
+
+       TRACE("Processing WRITE response" EVENT_TRACE(event));
+       if (unlikely(event->error)) {
+               error = -EIO;
+
+               ERROR("Write response include error(s) %d\n",
+                       event->error);
+       }
+
+       return error;
+}
+
+static int muenblk_client_read_resp(struct muenblk_device *dev,
+       struct bio *bio, struct muenblk_event *event)
+{
+       struct bio_vec bvec;
+       int error = 0;
+       ssize_t size = 0;
+
+       TRACE("Processing READ response" EVENT_TRACE(event));
+
+       if (unlikely(event->error)) {
+               error = -EIO;
+
+               ERROR("Read response include error(s) %d\n",
+                       event->error);
+
+               goto finish;
+       }
+
+       bvec = bio_iovec(bio);
+       size = muenblk_client_read_segment(dev, &bvec, event->data);
+
+       if (size < 0)
+               return size;
+
+       BUG_ON(size != bvec.bv_len);
+
+finish:
+       return error;
+}
+
+static int muenblk_client_read_req(struct muenblk_device *dev, struct bio *bio)
+{
+       int error = 0;
+       sector_t sector = bio->bi_iter.bi_sector >> 3;
+
+       dev->req.event->type = MUENBLK_EVENT_READ;
+       dev->req.event->id = sector;
+       dev->req.event->error = 0;
+       dev->req.event->priv = 0;
+
+       muenblk_client_send_server(dev, bio, dev->req.event);
+
+       return error;
+}
+
+static int muenblk_client_transfer_req(struct muenblk_device *dev,
+       struct bio *bio)
+{
+       int error = 0;
+       struct bio *split;
+
+       memset(dev->req.event, 0, sizeof(struct muenblk_event));
+
+       if (unlikely(bio == dev->dummy.size_bio)) {
+               TRACE("Received SIZE request -> processing command request"
+                               BIO_TRACE(bio));
+               error = muenblk_client_send_command_req(dev, bio,
+                               &bio->bi_iter.bi_sector, bio->bi_iter.bi_size,
+                               MUENBLK_COMMAND_SIZE);
+
+               if (unlikely(error))
+                       goto transfer_finish_inform_bio;
+
+               return 0;
+       }
+
+       if (unlikely(bio_end_sector(bio) > get_capacity(dev->disk))) {
+               ERROR("Failed to retrieve sector %lu of %lu -> out of bounds\n",
+                               bio_end_sector(bio), get_capacity(dev->disk));
+
+               error = -EIO;
+               goto transfer_finish_inform_bio;
+       }
+
+       if (bio->bi_iter.bi_size > EVENT_BLOCK_SIZE) {
+               TRACE("Large request: " BIO_TRACE(bio));
+               split = bio_split(bio, EVENT_BLOCK_SIZE / KERNEL_SECTOR_SIZE,
+                               GFP_NOIO, NULL);
+               bio_chain(split, bio);
+               generic_make_request(bio);
+               TRACE("Remaining     : " BIO_TRACE(bio));
+               bio = split;
+               TRACE("Now using     : " BIO_TRACE(bio));
+       }
+
+       switch (bio_op(bio)) {
+       case REQ_OP_READ:
+               TRACE("Received READ request -> processing" BIO_TRACE(bio));
+
+               error = muenblk_client_read_req(dev, bio);
+               if (!error)
+                       goto transfer_finish;
+               break;
+       case REQ_OP_WRITE:
+               if (likely(bio->bi_iter.bi_size > 0)) {
+                       TRACE("Received WRITE request -> processing"
+                                       BIO_TRACE(bio));
+
+                       error = muenblk_client_write_req(dev, bio);
+                       if (!error)
+                               goto transfer_finish;
+               } else if (unlikely(bio->bi_iter.bi_size == 0))
+                       TRACE("Received empty WRITE request -> ignoring"
+                                       BIO_TRACE(bio));
+               break;
+       default:
+               ERROR("Received UNKNOWN request -> not supported"
+                               BIO_TRACE(bio));
+               error = -EOPNOTSUPP;
+               break;
+       }
+
+transfer_finish_inform_bio:
+       bio->bi_error = error;
+       bio_endio(bio);
+
+transfer_finish:
+       return error;
+}
+
+static int muenblk_client_transfer_resp(struct muenblk_device *dev,
+       struct muenblk_event *event)
+{
+       struct bio *bio = NULL;
+       unsigned long flags = 0;
+       int error = 0;
+
+       spin_lock_irqsave(&dev->resp.lock, flags);
+
+#ifdef UNORDERED_RESPONSES
+       {
+               struct bio *prev_bio = NULL;
+               bool found = false;
+
+               bio_list_for_each(bio, &dev->resp.bio_list) {
+                       if ((uint64_t) bio == event->priv) {
+                               if (prev_bio)
+                                       prev_bio->bi_next = bio->bi_next;
+                               else
+                                       dev->resp.bio_list.head = bio->bi_next;
+
+                               if (dev->resp.bio_list.tail == bio)
+                                       dev->resp.bio_list.tail = prev_bio;
+
+                               found = true;
+                               break;
+                       }
+
+                       prev_bio = bio;
+               }
+
+               if (unlikely(!found)) {
+                       ERROR("BIO NOT FOUND - (%p, %p, %d, %llu)\n",
+                               dev->resp.bio_list.head,
+                               dev->resp.bio_list.tail,
+                               atomic_read(&dev->resp.in_flight),
+                               event->priv);
+                       BUG_ON(!found);
+               }
+       }
+#else
+       bio = bio_list_pop(&dev->resp.bio_list);
+#endif
+
+       BUG_ON(!bio);
+
+       atomic_dec(&dev->resp.in_flight);
+
+       spin_unlock_irqrestore(&dev->resp.lock, flags);
+
+       switch (event->type) {
+       case MUENBLK_EVENT_READ:
+               error = muenblk_client_read_resp(dev, bio, event);
+               break;
+       case MUENBLK_EVENT_WRITE:
+               error = muenblk_client_write_resp(dev, bio, event);
+               break;
+       case MUENBLK_EVENT_COMMAND:
+               error = muenblk_client_command_resp(dev, bio, event);
+               break;
+       default:
+               BUG();
+               break;
+       }
+
+       if (unlikely(muenblk_is_dummy_bio(dev, bio)))
+               goto transfer_finish;
+
+       bio->bi_error = error;
+       bio_endio(bio);
+
+transfer_finish:
+       if (unlikely(atomic_read(&dev->resp.in_flight)
+                               < dev->req.channel_queue_size))
+               wake_up(&dev->flow_wq);
+
+       return error;
+}
+
+static int muenblk_client_resp_thread(void *data)
+{
+       struct muenblk_device *dev = data;
+
+       TRACE("Entering ResponseThread\n");
+       muenblk_read_packets(dev, &dev->resp, &muenblk_client_transfer_resp,
+                       false);
+       TRACE("Leaving ResponseThread\n");
+
+       return 0;
+}
+
+static int muenblk_client_req_thread(void *data)
+{
+       struct muenblk_device *dev = data;
+       struct bio *bio;
+       unsigned long flags = 0;
+       int error = 0;
+
+       TRACE("Entering RequestThread\n");
+
+       while (!kthread_should_stop()) {
+               error = wait_event_interruptible(dev->req.wq,
+                       atomic_read(&dev->req.in_flight));
+
+               if (unlikely(kthread_should_stop())) {
+                       TRACE("RequestThread got STOP SIGNAL -> Terminating\n");
+                       goto out_thread;
+               }
+
+               if (unlikely(error == -ERESTARTSYS || error))
+                       continue;
+
+               spin_lock_irqsave(&dev->req.lock, flags);
+
+               bio = bio_list_pop(&dev->req.bio_list);
+               atomic_dec(&dev->req.in_flight);
+
+               spin_unlock_irqrestore(&dev->req.lock, flags);
+
+               BUG_ON(!bio);
+
+               TRACE("RequestThread processing request" BIO_TRACE(bio));
+
+               error = muenblk_client_transfer_req(dev, bio);
+
+               if (unlikely(error))
+                       ERROR("RequestThread processing request with error(s) %d\n", error);
+       }
+
+out_thread:
+       TRACE("Leaving RequestThread\n");
+
+       return 0;
+}
+
+static blk_qc_t muenblk_client_make_request(struct request_queue *queue,
+       struct bio *bio)
+{
+       struct muenblk_device *dev = queue->queuedata;
+       unsigned long flags = 0;
+
+       BUG_ON(!dev);
+       BUG_ON(!(bio_op(bio) == REQ_OP_READ || bio_op(bio) == REQ_OP_WRITE));
+
+       TRACE("Adding request into request queue" BIO_TRACE(bio));
+
+       spin_lock_irqsave(&dev->req.lock, flags);
+
+       bio_list_add(&dev->req.bio_list, bio);
+
+       atomic_inc(&dev->req.in_flight);
+
+       spin_unlock_irqrestore(&dev->req.lock, flags);
+
+       wake_up_interruptible(&dev->req.wq);
+
+       return BLK_QC_T_NONE;
+}
+
+static int muenblk_client_transfer_dummy_bio(struct muenblk_device *dev,
+       struct bio *dummy_bio)
+{
+       int error = 0;
+
+       TRACE("Sending dummy bio command %d request to server\n",
+               dummy_bio->bi_flags);
+
+       spin_lock(&dev->wq.lock);
+
+       muenblk_client_make_request(dev->queue, dummy_bio);
+
+       TRACE("Waiting for dummy bio command response\n");
+
+       while (1) {
+               error = wait_event_interruptible_locked(dev->wq,
+                       dummy_bio->bi_opf != 0);
+
+               if (unlikely(error == -ERESTARTSYS))
+                       continue;
+
+               if (unlikely(error < 0 || error > 0)) {
+                       ERROR("Dummy BIO transfer received signal %d\n",
+                               error);
+
+                       break;
+               }
+
+               error = dummy_bio->bi_vcnt;
+
+               TRACE("Received dummy bio event response with error %d\n",
+                               error);
+
+               if (error)
+                       ERROR("Received dummy bio event IO_ERROR %d\n", error);
+
+               break;
+       }
+
+       spin_unlock(&dev->wq.lock);
+
+       return error;
+}
+
+static int muenblk_client_getgeo(struct block_device *bdev,
+       struct hd_geometry *geo)
+{
+       struct muenblk_device *dev = bdev->bd_disk->private_data;
+       int error = 0;
+
+       geo->cylinders = (dev->disk_size & ~0x3f) >> 6;
+       geo->heads = 4;
+       geo->sectors = 16;
+       geo->start = 4;
+
+       return error;
+}
+
+static int muenblk_client_ioctl(struct block_device *bdev, fmode_t mode,
+       unsigned int command, unsigned long arg)
+{
+       switch (command) {
+       case CDROM_GET_CAPABILITY:
+               /* Used by (busybox) fsck */
+               return -ENOTSUPP;
+       default:
+               TRACE("Received UNKNOWN IOCTL (0x%x)\n", command);
+               return -ENOTSUPP;
+       }
+
+       return 0;
+}
+
+static int muenblk_retrieve_size(struct muenblk_device *dev)
+{
+       int error = 0;
+
+       TRACE("Processing SIZE COMMAND\n");
+
+       mutex_lock(&dev->mutex);
+
+       dev->dummy.size_bio->bi_bdev = dev->bdev;
+       dev->dummy.size_bio->bi_flags = MUENBLK_COMMAND_SIZE;
+       dev->dummy.size_bio->bi_opf = 0;
+       dev->dummy.size_bio->bi_vcnt = 0;
+       dev->dummy.size_bio->bi_next = NULL;
+
+       dev->dummy.size_bio->bi_iter.bi_bvec_done = 0;
+       dev->dummy.size_bio->bi_iter.bi_idx = 0;
+       dev->dummy.size_bio->bi_iter.bi_sector = 0;
+       dev->dummy.size_bio->bi_iter.bi_size = 0;
+
+       error = muenblk_client_transfer_dummy_bio(dev, dev->dummy.size_bio);
+
+       if (!error) {
+               dev->disk_sector_count = dev->dummy.size_bio->bi_iter.bi_sector;
+               dev->disk_kernel_sector_count = dev->disk_sector_count;
+               dev->disk_kernel_start_sector = 0;
+               dev->disk_size = dev->disk_kernel_sector_count *
+                       KERNEL_SECTOR_SIZE;
+
+               TRACE("Received disk size (%llu, %llu, %llu)\n",
+                       dev->disk_size,
+                       dev->disk_sector_count,
+                       dev->disk_kernel_sector_count);
+       }
+
+       mutex_unlock(&dev->mutex);
+
+       return error;
+}
+
+static const struct block_device_operations muenblk_fops = {
+       .owner = THIS_MODULE,
+       .ioctl = muenblk_client_ioctl,
+       .getgeo = muenblk_client_getgeo,
+};
+
+static int muenblk_client_init_queue(struct muenblk_device *dev)
+{
+       int error = 0;
+
+       dev->queue = blk_alloc_queue(GFP_KERNEL);
+       if (!dev->queue) {
+               ERROR("Failed to allocate block device request queue\n");
+               error = -ENOMEM;
+               goto exit;
+       }
+
+       dev->queue->queuedata = dev;
+
+       blk_queue_make_request(dev->queue, muenblk_client_make_request);
+
+       blk_queue_write_cache(dev->queue, true, false);
+       queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, dev->queue);
+
+       TRACE("Successfully initialized queue\n");
+
+exit:
+       return error;
+}
+
+static int muenblk_client_format_disk_name(char *prefix, int index, char *buf,
+       int buflen)
+{
+       const int base = 'z' - 'a' + 1;
+       char *begin = buf + strlen(prefix);
+       char *end = buf + buflen;
+       char *p;
+       int unit;
+
+       p = end - 1;
+       *p = '\0';
+       unit = base;
+       do {
+               if (p == begin)
+                       return -EINVAL;
+               *--p = 'a' + (index % unit);
+               index = (index / unit) - 1;
+       } while (index >= 0);
+
+       memmove(begin, p, end - p);
+       memcpy(buf, prefix, strlen(prefix));
+
+       return 0;
+}
+
+static int muenblk_client_init_blkdev(struct muenblk_device *dev)
+{
+       int error = 0;
+
+       TRACE("Initializing block device...\n");
+
+       if (!dev) {
+               error = -EINVAL;
+               goto err_blk_out;
+       }
+
+       error = muenblk_client_format_disk_name(MUENBLK_DEV_NAME,
+               dev->disk_index, dev->disk_name, DISK_NAME_LEN);
+       if (error) {
+               ERROR("Failed to generate block device disk name\n");
+               goto err_blk_out;
+       }
+
+       dev->disk_major = register_blkdev(MUENBLK_DEFAULT_MAJOR,
+                       dev->disk_name);
+       if (dev->disk_major <= 0) {
+               ERROR("Failed to register block device\n");
+               error = -EIO;
+               goto err_blk_out;
+       }
+
+       dev->disk = alloc_disk(MUENBLK_MINOR_COUNT);
+       if (!dev->disk) {
+               ERROR("Failed to allocate block device disk\n");
+
+               error = -ENOMEM;
+               goto err_blk_cleanup_dev;
+       }
+
+       dev->disk->major = dev->disk_major;
+       dev->disk->first_minor = MUENBLK_MINOR_FIRST;
+       dev->disk->minors =  MUENBLK_MINOR_COUNT;
+       dev->disk->queue = dev->queue;
+       dev->disk->private_data = dev;
+       dev->disk->flags = 0;
+       dev->disk->fops = &muenblk_fops;
+
+       memcpy(dev->disk->disk_name, dev->disk_name, DISK_NAME_LEN);
+       set_capacity(dev->disk, dev->disk_kernel_sector_count);
+
+       add_disk(dev->disk);
+
+       dev->mode = 0;
+       dev->bio_set = NULL;
+       dev->page_pool = NULL;
+
+       if (error) {
+               ERROR("Failed to initialize block device\n");
+
+               error = -EIO;
+               goto err_blk_cleanup_disk;
+       }
+
+       TRACE("Successfully initialized block device %s (%u)\n",
+               dev->disk_name, dev->disk_index);
+
+       return 0;
+
+
+err_blk_cleanup_disk:
+       del_gendisk(dev->disk);
+       put_disk(dev->disk);
+
+err_blk_cleanup_dev:
+       unregister_blkdev(dev->disk_major, dev->disk_name);
+       dev->disk_major = 0;
+       dev->disk_minor = 0;
+
+err_blk_out:
+       return error;
+}
+
+static void muenblk_client_cleanup_queue(struct muenblk_device *dev)
+{
+       if (dev && dev->queue)
+               blk_cleanup_queue(dev->queue);
+       TRACE("Queue cleanup successful\n");
+}
+
+static void muenblk_client_cleanup_blkdev(struct muenblk_device *dev)
+{
+       if (!dev)
+               return;
+
+       if (dev->disk) {
+               del_gendisk(dev->disk);
+               put_disk(dev->disk);
+               dev->disk = NULL;
+       }
+
+       if (dev->disk_major) {
+               unregister_blkdev(dev->disk_major, dev->disk_name);
+               dev->disk_major = 0;
+               dev->disk_minor = 0;
+       }
+
+       TRACE("Block device cleanup succesfull\n");
+
+}
+
+static void muenblk_client_exit_dev(struct muenblk_device *dev)
+{
+       if (dev) {
+               muenblk_cleanup_shm(dev, false);
+               muenblk_client_cleanup_blkdev(dev);
+               muenblk_client_cleanup_queue(dev);
+               muenblk_cleanup_dummy_bio(dev);
+
+               kfree(dev);
+       }
+}
+
+static void muenblk_client_exit(void)
+{
+       int i = 0;
+
+       for (i = 0; i < dev_name_count; i++) {
+               muenblk_client_exit_dev(blk_dev[i]);
+               blk_dev[i] = NULL;
+       }
+}
+
+static void muenblk_client_init_worker(struct work_struct *work)
+{
+       int error = 0;
+       struct muenblk_device *dev = container_of(work, struct muenblk_device,
+               init_work);
+
+       error = muenblk_retrieve_size(dev);
+       if (error) {
+               ERROR("Failed to retrieve disk size");
+               ERROR("FATAL ERROR OCCURRED AT DISK %s (%d) -> EXIT",
+                       dev->disk_name, dev->disk_index);
+
+               muenblk_client_exit_dev(dev);
+       }
+
+       error = muenblk_client_init_blkdev(dev);
+       if (error) {
+               ERROR("Failed to initialize disk");
+               ERROR("FATAL ERROR OCCURRED AT DISK %s (%d) -> EXIT",
+                       dev->disk_name, dev->disk_index);
+
+               muenblk_client_exit_dev(dev);
+       }
+}
+
+static int __init muenblk_client_init(void)
+{
+       int i = 0;
+       unsigned int len = 0;
+       int error = 0;
+
+       struct muenblk_device *dev = NULL;
+
+       if (!dev_name_count)
+               return -EINVAL;
+
+       for (i = 0; i < dev_name_count; i++) {
+               dev = kzalloc(sizeof(struct muenblk_device), GFP_KERNEL);
+
+               if (!dev) {
+                       error = -ENOMEM;
+                       goto err_out;
+               }
+
+               mutex_init(&dev->mutex);
+               init_waitqueue_head(&dev->wq);
+               init_waitqueue_head(&dev->flow_wq);
+
+               INIT_WORK(&dev->init_work, muenblk_client_init_worker);
+
+               dev->req.thread_fn = muenblk_client_req_thread;
+               dev->resp.thread_fn = muenblk_client_resp_thread;
+               dev->poll_interval = poll_interval;
+               dev->disk_major = 0;
+               dev->disk_minor = 0;
+               dev->disk_index = i;
+
+               len = strlen(req_shm_name[i]);
+               if (len >= 64)
+                       goto err_cleanup;
+
+               strncpy(dev->req.channel_name, req_shm_name[i], len);
+               dev->req.channel_name[len] = 0;
+
+               len = strlen(resp_shm_name[i]);
+               if (len >= 64)
+                       goto err_cleanup;
+
+               strncpy(dev->resp.channel_name, resp_shm_name[i], len);
+               dev->resp.channel_name[len] = 0;
+
+               if (req_shm_protocol[i] && strlen(req_shm_protocol[i]) > 0) {
+                       error = kstrtoull(req_shm_protocol[i], 16,
+                                       &dev->req.channel_protocol);
+                       if (error) {
+                               ERROR("Invalid request protocol '%s'\n",
+                                               req_shm_protocol[i]);
+                               goto err_cleanup;
+                       }
+               } else
+                       dev->req.channel_protocol = 0;
+
+               if (resp_shm_protocol[i] && strlen(resp_shm_protocol[i]) > 0) {
+                       error = kstrtoull(resp_shm_protocol[i], 16,
+                                       &dev->resp.channel_protocol);
+                       if (error) {
+                               ERROR("Invalid response protocol '%s'\n",
+                                               req_shm_protocol[i]);
+                               goto err_cleanup;
+                       }
+               } else
+                       dev->resp.channel_protocol = 0;
+
+               error = muenblk_init_dummy_bio(dev);
+               if (error)
+                       goto err_cleanup;
+
+               error = muenblk_client_init_queue(dev);
+               if (error)
+                       goto err_cleanup_bio;
+
+               error = muenblk_init_shm(dev, false);
+               if (error)
+                       goto err_cleanup_queue;
+
+               dev->disk = NULL;
+               blk_dev[i] = dev;
+
+               if (error)
+                       goto err_cleanup_shm;
+       }
+
+       TRACE("Successfully initialized module\n");
+
+       return 0;
+
+err_cleanup_shm:
+       muenblk_cleanup_shm(dev, false);
+
+err_cleanup_queue:
+       muenblk_client_cleanup_queue(dev);
+
+err_cleanup_bio:
+       muenblk_cleanup_dummy_bio(dev);
+
+err_cleanup:
+       kfree(dev);
+
+err_out:
+       muenblk_client_exit();
+
+       return error;
+
+}
+
+module_init(muenblk_client_init);
+module_exit(muenblk_client_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Dennis Wassenberg <dennis.wassenberg@secunet.com>");
+MODULE_DESCRIPTION("Muen SK block device driver (client)");
+
+module_param_array(req_shm_name, charp, &dev_name_count, 0444);
+MODULE_PARM_DESC(req_shm_name, "List of request shared memory region names.");
+
+module_param_array(resp_shm_name, charp, NULL, 0444);
+MODULE_PARM_DESC(resp_shm_name, "List of response shared memory region names.");
+
+module_param_array(req_shm_protocol, charp, NULL, 0444);
+MODULE_PARM_DESC(req_shm_protocol, "List of request shared memory protocols.");
+
+module_param_array(resp_shm_protocol, charp, NULL, 0444);
+MODULE_PARM_DESC(resp_shm_protocol, "List of response shared memory protocol.");
+
+module_param(poll_interval, uint, 0444);
+MODULE_PARM_DESC(poll_interval, "Polling interval.");
diff --git a/common.c b/common.c
new file mode 100644 (file)
index 0000000..e461b53
--- /dev/null
+++ b/common.c
@@ -0,0 +1,283 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * Muen block I/O device driver.
+ *
+ * Copyright (C) 2018  secunet Security Networks AG
+ *
+ */
+
+#define KBUILD_MODNAME "muenblock_common"
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/device.h>
+#include <linux/kthread.h>
+#include <linux/printk.h>
+#include <linux/random.h>
+
+#include "common.h"
+#include "log.h"
+
+static void muenblk_cleanup_channel(
+       struct muenblk_device_channel *chan, bool writer)
+{
+       if (chan) {
+               if (chan->thread) {
+                       atomic_inc(&chan->in_flight);
+                       kthread_stop(chan->thread);
+                       atomic_dec(&chan->in_flight);
+                       chan->thread = NULL;
+               }
+
+               if (writer && chan->channel_data
+                               && chan->channel_element_size > 0)
+                       muen_channel_deactivate(chan->channel_data);
+
+               chan->channel_element_size = 0;
+
+               iounmap(chan->channel_data);
+               chan->channel_data = NULL;
+               chan->thread_fn = NULL;
+       }
+       TRACE("Channel cleanup successful\n");
+}
+
+int muenblk_init_channel(
+       struct muenblk_device_channel *chan,
+       struct muenblk_device *dev,
+       bool writer)
+{
+       int error = 0;
+       u64 epoch;
+
+       if (!chan) {
+               ERROR("Failed to retrieve channel\n");
+               error = -EINVAL;
+               goto err_out;
+       }
+       if (!chan->channel_info.address) {
+               ERROR("Failed to retrieve channel information\n");
+               error = -EINVAL;
+               goto err_out;
+       }
+       if (chan->channel_info.size < sizeof(struct muenblk_event)) {
+               ERROR("Channel size of %llu bytes too small, minimum is %lu\n", chan->channel_info.size, sizeof(struct muenblk_event));
+               error = -EINVAL;
+               goto err_out;
+       }
+
+       if (writer && !chan->channel_info.writable) {
+               ERROR("Channel is not writeable\n");
+               error = -EPERM;
+               goto err_out;
+       }
+
+       chan->event = kzalloc(sizeof(struct muenblk_event),
+                       GFP_KERNEL);
+       if (!chan->event) {
+               ERROR("Failed to allocate event memory\n");
+               error = -ENOMEM;
+               goto err_event_out;
+       }
+
+       bio_list_init(&chan->bio_list);
+
+       chan->channel_element_size = sizeof(struct muenblk_event);
+       chan->channel_queue_size =
+               (chan->channel_info.size
+                - sizeof(struct muchannel_header))
+               / chan->channel_element_size;
+
+       chan->channel_data = ioremap_cache(
+                       chan->channel_info.address,
+                       chan->channel_info.size);
+       if (!chan->channel_data) {
+               ERROR("Failed to allocate channel memory\n");
+               error = -ENOMEM;
+               goto err_state_out;
+       }
+
+       if (writer) {
+               get_random_bytes(&epoch, sizeof(epoch));
+               muen_channel_init_writer(
+                               chan->channel_data,
+                               chan->channel_protocol,
+                               chan->channel_element_size,
+                               chan->channel_info.size,
+                               epoch);
+       } else
+               muen_channel_init_reader(&chan->reader, chan->channel_protocol);
+
+       chan->thread = kthread_create(chan->thread_fn, dev,
+                       writer ? "muenblk_writer" : "muenblk_reader");
+
+       if (IS_ERR(chan->thread)) {
+               ERROR("Failed to allocate shm thread\n");
+
+               error = PTR_ERR(chan->thread);
+               goto err_thread_out;
+       }
+
+       spin_lock_init(&chan->lock);
+       atomic_set(&chan->in_flight, 0);
+       init_waitqueue_head(&chan->wq);
+
+       wake_up_process(chan->thread);
+
+       TRACE("Successfully initialized %s channel (proto 0x%llx, element size %lu, queue size %lu)\n",
+                       writer ? "writer" : "reader",
+                       chan->channel_protocol,
+                       chan->channel_element_size,
+                       chan->channel_queue_size);
+
+       return 0;
+
+err_thread_out:
+       chan->thread = NULL;
+
+err_state_out:
+       chan->channel_element_size = 0;
+
+       kfree(chan->event);
+
+err_event_out:
+       chan->event = NULL;
+
+err_out:
+       return error;
+}
+
+void muenblk_read_packets(struct muenblk_device *dev,
+               struct muenblk_device_channel *chan,
+               int (*transfer_func)(struct muenblk_device *dev,
+                       struct muenblk_event *event),
+               bool server)
+{
+       enum muchannel_reader_result result = MUCHANNEL_SUCCESS;
+       int error = 0;
+
+       while (!kthread_should_stop()) {
+               result = muen_channel_read(chan->channel_data,
+                               &chan->reader,
+                               chan->event);
+
+               switch (result) {
+               case MUCHANNEL_EPOCH_CHANGED:
+                       TRACE("Channel '%s' received new epoch\n", chan->channel_name);
+                       if (!server) {
+                               muen_channel_drain(chan->channel_data,
+                                               &chan->reader);
+                               schedule_work(&dev->init_work);
+                       }
+                       break;
+
+               case MUCHANNEL_SUCCESS:
+                       TRACE("Channel '%s' received data\n", chan->channel_name);
+                       TRACE(EVENT_TRACE(chan->event));
+
+                       error = transfer_func(dev, chan->event);
+                       if (unlikely(error))
+                               ERROR("Channel '%s' processing request with error(s) %d\n", chan->channel_name, error);
+
+                       continue;
+
+               case MUCHANNEL_NO_DATA:
+               case MUCHANNEL_INACTIVE:
+                       goto schedule;
+
+               case MUCHANNEL_OVERRUN_DETECTED:
+               case MUCHANNEL_INCOMPATIBLE_INTERFACE:
+               default:
+                       ERROR("Channel '%s' overrun or incompatible interface: %d\n",
+                                       chan->channel_name, result);
+                       return;
+               }
+
+schedule:
+               schedule_timeout_interruptible(dev->poll_interval);
+       }
+}
+
+int muenblk_init_dummy_bio(struct muenblk_device *dev)
+{
+       int error = 0;
+
+       dev->dummy.size_bio = kzalloc(sizeof(struct bio), GFP_KERNEL);
+
+       if (!dev->dummy.size_bio) {
+               ERROR("Failed create dummy bios\n");
+               muenblk_cleanup_dummy_bio(dev);
+               return -ENOMEM;
+       }
+
+       memset(dev->dummy.size_bio, 0, sizeof(struct bio));
+       dev->dummy.size_bio->bi_private = kzalloc(sizeof(
+                       struct muenblk_priv_data), GFP_KERNEL);
+
+       if (!dev->dummy.size_bio->bi_private) {
+               ERROR("Failed create dummy bio metadata\n");
+               muenblk_cleanup_dummy_bio(dev);
+               return -ENOMEM;
+       }
+
+       TRACE("Successfully initialized dummy bios\n");
+       return error;
+}
+
+void muenblk_cleanup_dummy_bio(struct muenblk_device *dev)
+{
+       if (dev->dummy.size_bio) {
+               kfree(dev->dummy.size_bio->bi_private);
+               kfree(dev->dummy.size_bio);
+               dev->dummy.size_bio = NULL;
+       }
+
+       TRACE("Dummy bio cleanup successful\n");
+}
+
+int muenblk_is_dummy_bio(struct muenblk_device *dev, struct bio *bio)
+{
+       return bio == dev->dummy.size_bio;
+}
+
+int muenblk_init_shm(struct muenblk_device *dev, bool server)
+{
+       int error = 0;
+
+       BUG_ON(!dev);
+
+       if (!muen_get_channel_info(dev->req.channel_name,
+                               &dev->req.channel_info)) {
+               ERROR("Failed to determine shm request channel\n");
+               return -EINVAL;
+       }
+       if (!muen_get_channel_info(dev->resp.channel_name,
+                               &dev->resp.channel_info)) {
+               ERROR("Failed to determine shm response channel\n");
+               return -EINVAL;
+       }
+       if (dev->resp.channel_info.size != dev->req.channel_info.size) {
+               ERROR("Request and response channel sizes differ, must be of same size");
+               return -EINVAL;
+       }
+
+       error = muenblk_init_channel(&dev->req, dev, !server);
+       if (error)
+               return error;
+       error = muenblk_init_channel(&dev->resp, dev, server);
+       if (error) {
+               muenblk_cleanup_channel(&dev->req, true);
+               return error;
+       }
+
+       TRACE("Successfully initialized shared memory channels\n");
+
+       return 0;
+}
+
+void muenblk_cleanup_shm(struct muenblk_device *dev, bool server)
+{
+       muenblk_cleanup_channel(&dev->req, !server);
+       muenblk_cleanup_channel(&dev->resp, server);
+       TRACE("Shared memory channel cleanup successful\n");
+}
diff --git a/common.h b/common.h
new file mode 100644 (file)
index 0000000..6dbe564
--- /dev/null
+++ b/common.h
@@ -0,0 +1,133 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Muen block I/O device driver.
+ *
+ * Copyright (C) 2018  secunet Security Networks AG
+ *
+ */
+
+#ifndef MUENBLK_COMMON_H_
+#define MUENBLK_COMMON_H_
+
+#define EVENT_BLOCK_SIZE       4096
+#define KERNEL_SECTOR_SIZE     512
+#define MUENBLK_DEFAULT_MAJOR  0
+#define MUENBLK_MINOR_FIRST    0
+#define MUENBLK_MINOR_COUNT    16
+#define MUENBLK_DEV_NAME       "muendisk"
+
+#include <linux/bio.h>
+#include <linux/genhd.h>
+#include <linux/workqueue.h>
+
+#include <muen/sinfo.h>
+#include <muen/reader.h>
+#include <muen/writer.h>
+
+enum muenblk_command {
+       MUENBLK_COMMAND_SYNC = 0,
+       MUENBLK_COMMAND_SIZE
+};
+
+enum muenblk_event_type {
+       MUENBLK_EVENT_READ = 0,
+       MUENBLK_EVENT_WRITE,
+       MUENBLK_EVENT_COMMAND,
+       MUENBLK_EVENT_LAST
+};
+
+struct muenblk_event {
+       enum muenblk_event_type         type;
+       int32_t                         error;
+       uint64_t                        id;
+       uint64_t                        priv;
+       char                            data[EVENT_BLOCK_SIZE];
+};
+
+struct muenblk_priv_data;
+
+struct muenblk_device_channel {
+       char                            channel_name[64];
+       struct muen_channel_info        channel_info;
+
+       struct muchannel                *channel_data;
+       struct muchannel_reader         reader;
+
+       size_t                          channel_element_size;
+       size_t                          channel_queue_size;
+       uint64_t                        channel_protocol;
+
+       struct task_struct              *thread;
+       int                             (*thread_fn)(void *data);
+       wait_queue_head_t               wq;
+       spinlock_t                      lock;
+       atomic_t                        in_flight;
+
+       struct bio_list                 bio_list;
+       struct muenblk_event            *event;
+};
+
+struct muenblk_dummy_bio {
+       struct bio                      *size_bio;
+};
+
+struct muenblk_device {
+       char                            disk_name[DISK_NAME_LEN];
+       struct gendisk                  *disk;
+       struct block_device             *bdev;
+       struct mutex                    mutex;
+       wait_queue_head_t               wq;
+       wait_queue_head_t               flow_wq;
+       struct work_struct              init_work;
+
+       struct muenblk_dummy_bio        dummy;
+       long                            poll_interval;
+
+       uint64_t                        disk_size;
+       uint64_t                        disk_sector_count;
+       uint64_t                        disk_kernel_sector_count;
+       uint64_t                        disk_kernel_start_sector;
+       uint32_t                        disk_major;
+       uint8_t                         disk_minor;
+       uint8_t                         disk_index;
+
+       struct muenblk_device_channel   req;
+       struct muenblk_device_channel   resp;
+
+       struct request_queue            *queue;
+       fmode_t                         mode;
+
+       struct bio_set                  *bio_set;
+       mempool_t                       *page_pool;
+       mempool_t                       *metadata_pool;
+};
+
+struct muenblk_priv_data {
+       struct muenblk_device           *dev;
+       uint64_t                        priv;
+       int32_t                         error;
+       int32_t                         reserved;
+};
+
+int muenblk_init_channel(struct muenblk_device_channel *muenblk_dev_channel,
+                        struct muenblk_device *dev, bool writer);
+
+/*
+ * Read packets from given channel until an error occurs or calling kthread
+ * should stop
+ */
+void muenblk_read_packets(struct muenblk_device *dev,
+                         struct muenblk_device_channel *channel,
+                         int (*transfer_func)(struct muenblk_device *dev,
+                             struct muenblk_event *event),
+                         bool server);
+
+int muenblk_init_dummy_bio(struct muenblk_device *dev);
+void muenblk_cleanup_dummy_bio(struct muenblk_device *dev);
+int muenblk_is_dummy_bio(struct muenblk_device *dev, struct bio *bio);
+
+int muenblk_init_shm(struct muenblk_device *dev, bool server);
+void muenblk_cleanup_shm(struct muenblk_device *dev, bool server);
+
+#endif /* MUENBLK_COMMON_H_ */
diff --git a/log.h b/log.h
new file mode 100644 (file)
index 0000000..1738316
--- /dev/null
+++ b/log.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Muen block I/O device driver.
+ *
+ * Copyright (C) 2018  secunet Security Networks AG
+ *
+ */
+
+#ifndef MUENBLK_LOG_H_
+#define MUENBLK_LOG_H_
+
+//#define DEBUG 1
+
+#define ERROR(args ...)        pr_err(args)
+#define INFO(args ...)  pr_info(args)
+
+#ifdef DEBUG
+       #define BIO_TRACE(bio) " - (bio: %u, %u, %lu, %u, %u," \
+           " %u, %u, %p, %p)\n", bio_op(bio), bio->bi_vcnt, \
+           bio->bi_iter.bi_sector, bio->bi_iter.bi_size, bio->bi_iter.bi_idx, \
+           bio->bi_iter.bi_bvec_done, bio->bi_flags, bio->bi_bdev, bio
+
+       #define EVENT_TRACE(event) " - (event: %d, %llu, %d, %llx, %p)\n", \
+           event->type, event->id, event->error, event->priv, event
+
+       #define COPY_TRACE(dst, src, size) " - (copy: %p, %p, %llu)\n", dst, \
+           src, (uint64_t) size
+
+       #define TRACE(args ...) pr_info(args)
+#else
+       #define BIO_TRACE(bio)
+       #define EVENT_TRACE(event)
+       #define TRACE(args ...)
+#endif
+
+#endif /* MUENBLK_LOG_H_ */
diff --git a/server.c b/server.c
new file mode 100644 (file)
index 0000000..cb6f42c
--- /dev/null
+++ b/server.c
@@ -0,0 +1,803 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * Muen block I/O device driver.
+ *
+ * Copyright (C) 2018  secunet Security Networks AG
+ *
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/init.h>
+#include <linux/genhd.h>
+#include <linux/hdreg.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/mutex.h>
+#include <linux/major.h>
+#include <linux/blkdev.h>
+#include <linux/bio.h>
+#include <linux/kernel.h>
+#include <linux/kthread.h>
+#include <linux/fs.h>
+#include <linux/types.h>
+
+#include "common.h"
+#include "log.h"
+
+static char *req_shm_name[MUENBLK_MINOR_COUNT];
+static char *resp_shm_name[MUENBLK_MINOR_COUNT];
+
+static char *req_shm_protocol[MUENBLK_MINOR_COUNT];
+static char *resp_shm_protocol[MUENBLK_MINOR_COUNT];
+
+static int dev_name_count;
+static char *device_name[MUENBLK_MINOR_COUNT];
+
+static unsigned int poll_interval = 1;
+
+static struct muenblk_device *blk_dev[MUENBLK_MINOR_COUNT];
+
+static void muenblk_server_cb_endio(struct bio *bio);
+static void muenblk_server_queue_resp(struct muenblk_device *dev,
+       struct bio *bio);
+
+static void cleanup_bio(struct bio *bio)
+{
+       if (bio) {
+               struct muenblk_priv_data *metadata = NULL;
+
+               metadata = bio->bi_private;
+
+               if (metadata)
+                       mempool_free(metadata, metadata->dev->metadata_pool);
+
+               bio->bi_private = NULL;
+               bio_put(bio);
+       }
+}
+
+static void muenblk_server_send_resp(struct muenblk_device *dev,
+       struct muenblk_event *event)
+{
+       TRACE("Sending response event to client" EVENT_TRACE(event));
+       muen_channel_write(dev->resp.channel_data, event);
+}
+
+static int muenblk_server_command_req(struct muenblk_device *dev,
+       struct muenblk_event *event, struct bio *bio)
+{
+       int error = 0;
+
+       switch (event->id) {
+       case MUENBLK_COMMAND_SYNC:
+               bio->bi_opf = REQ_OP_FLUSH;
+
+               break;
+
+       case MUENBLK_COMMAND_SIZE:
+               bio->bi_opf = REQ_OP_READ;
+               bio->bi_iter.bi_sector = 0;
+               bio->bi_iter.bi_idx = 0;
+               bio->bi_iter.bi_size = 0;
+               bio->bi_iter.bi_bvec_done = 0;
+               bio->bi_bdev = NULL;
+
+               break;
+
+       default:
+               TRACE("Unknown command id %llu", event->id);
+
+               break;
+       }
+
+       return error;
+}
+
+static int muenblk_server_create_bio_mem(struct muenblk_device *dev,
+       struct muenblk_event *event, struct bio **bio_ptr, int *dummy_bio)
+{
+       int error = 0;
+       struct muenblk_priv_data *metadata = NULL;
+
+       *bio_ptr = NULL;
+
+       if (unlikely(event->type == MUENBLK_EVENT_COMMAND)) {
+               switch (event->id) {
+               case MUENBLK_COMMAND_SYNC:
+                       *dummy_bio = false;
+                       // nothing to do
+
+                       break;
+
+               case MUENBLK_COMMAND_SIZE:
+                       *dummy_bio = true;
+                       *bio_ptr = dev->dummy.size_bio;
+
+                       break;
+
+               default:
+                       ERROR("Failed to create bio memory for UNKNOWN command\n");
+
+                       *dummy_bio = false;
+                       error = -EINVAL;
+                       goto exit;
+               }
+       }
+
+       if (likely(!*bio_ptr)) {
+               *bio_ptr = bio_alloc_bioset(GFP_ATOMIC, 1, dev->bio_set);
+
+               if (unlikely(!*bio_ptr)) {
+                       ERROR("Failed to allocate bio from bioset\n");
+                       error = -ENOMEM;
+                       goto exit;
+               }
+
+               metadata = mempool_alloc(dev->metadata_pool, GFP_KERNEL);
+               if (unlikely(!metadata)) {
+                       ERROR("Failed to allocate metadata from mempool\n");
+                       error = -ENOMEM;
+                       goto err_cleanup_bio;
+               }
+
+               (*bio_ptr)->bi_private = metadata;
+       }
+
+       return 0;
+
+err_cleanup_bio:
+       cleanup_bio(*bio_ptr);
+       *bio_ptr = NULL;
+
+exit:
+       return error;
+}
+
+static int muenblk_server_create_bio_page(struct muenblk_device *dev,
+       struct muenblk_event *event, struct bio *bio)
+{
+       struct page *page;
+       int size = 0;
+       int error = 0;
+
+       page = mempool_alloc(dev->page_pool, GFP_NOIO);
+
+       if (unlikely(!page)) {
+               ERROR("Failed to allocate bio page from mempool\n");
+
+               error = -ENOMEM;
+               goto exit;
+       }
+
+       size = bio_add_page(bio, page, EVENT_BLOCK_SIZE, 0);
+
+       if (unlikely(size != EVENT_BLOCK_SIZE)) {
+               ERROR("Failed to add page to bio" BIO_TRACE(bio));
+
+               error = -ENOMEM;
+               goto error_cleanup_page;
+       }
+
+       TRACE("Bound page %p to bio %p with %d bytes\n", page, bio, size);
+
+       return 0;
+
+error_cleanup_page:
+       mempool_free(page, dev->page_pool);
+
+exit:
+       return error;
+}
+
+static int muenblk_server_init_bio(struct muenblk_device *dev,
+       struct muenblk_event *event, struct bio **bio_ptr)
+{
+       int error = 0;
+       int dummy_bio = 0;
+       struct bio *bio = NULL;
+       struct muenblk_priv_data *metadata = NULL;
+
+       error = muenblk_server_create_bio_mem(dev, event, bio_ptr, &dummy_bio);
+
+       if (error)
+               goto exit;
+
+       bio = *bio_ptr;
+
+       metadata = bio->bi_private;
+       metadata->dev = dev;
+       metadata->error = 0;
+       metadata->priv = event->priv;
+
+       bio->bi_bdev = dev->bdev;
+       bio->bi_end_io = muenblk_server_cb_endio;
+       bio->bi_iter.bi_sector = event->id << 3;
+       bio->bi_iter.bi_size = 0;
+
+       switch (event->type) {
+       case MUENBLK_EVENT_READ:
+       case MUENBLK_EVENT_WRITE:
+               bio->bi_opf = event->type == MUENBLK_EVENT_READ ?
+                       REQ_OP_READ : REQ_OP_WRITE;
+               error = muenblk_server_create_bio_page(dev, event, bio);
+               break;
+       case MUENBLK_EVENT_COMMAND:
+               error = muenblk_server_command_req(dev, event, bio);
+               break;
+       default:
+               // FIXME
+               TRACE("Unknown event type %u", event->type);
+               error = -EINVAL;
+               break;
+       }
+
+       if (unlikely(error))
+               goto error_cleanup_bio;
+
+       return 0;
+
+error_cleanup_bio:
+       if (!dummy_bio)
+               cleanup_bio(bio);
+
+exit:
+       return error;
+}
+
+static int muenblk_server_write_req(struct muenblk_device *dev,
+       struct muenblk_event *event, struct bio *bio)
+{
+       struct bio_vec *bvec = NULL;
+       int i = 0;
+       int error = 0;
+
+       bio_for_each_segment_all(bvec, bio, i) {
+               void *mapped_page = kmap_atomic(bvec->bv_page);
+
+               TRACE("Writing segment data from client request event to page"
+                               COPY_TRACE(mapped_page, event->data,
+                                       EVENT_BLOCK_SIZE));
+
+               memcpy(mapped_page, event->data, EVENT_BLOCK_SIZE);
+
+//             print_hex_dump(KERN_INFO, "server event", DUMP_PREFIX_OFFSET,
+//                     16, 1, event->data, EVENT_BLOCK_SIZE, true);
+//             print_hex_dump(KERN_INFO, "server page", DUMP_PREFIX_OFFSET,
+//                     16, 1, mapped_page, EVENT_BLOCK_SIZE, true);
+
+               kunmap_atomic(mapped_page);
+       }
+
+       return error;
+}
+
+static int muenblk_server_transfer_req(struct muenblk_device *dev,
+       struct muenblk_event *event)
+{
+       struct bio *bio = NULL;
+       int error = 0;
+
+       error = muenblk_server_init_bio(dev, event, &bio);
+       if (unlikely(error))
+               goto exit;
+
+       if (unlikely(muenblk_is_dummy_bio(dev, bio))) {
+               muenblk_server_queue_resp(dev, bio);
+               goto exit;
+       } else if (bio_op(bio) == REQ_OP_WRITE) {
+               error = muenblk_server_write_req(dev, event, bio);
+       } else {
+               BUG_ON(bio->bi_opf);
+       }
+
+       TRACE("Generating block request for bio" BIO_TRACE(bio));
+       generic_make_request(bio);
+
+exit:
+       return error;
+}
+
+static int muenblk_server_read_resp(struct muenblk_device *dev,
+       struct muenblk_event *event, struct bio *bio)
+{
+       struct bio_vec *bvec = NULL;
+       int i = 0;
+       int error = 0;
+
+       TRACE("Processing READ response" BIO_TRACE(bio));
+
+       event->type = MUENBLK_EVENT_READ;
+       event->id = ((bio->bi_iter.bi_sector -
+                               dev->disk_kernel_start_sector) >> 3) - 1;
+
+       bio_for_each_segment_all(bvec, bio, i) {
+               void *mapped_page = kmap_atomic(bvec->bv_page);
+
+               TRACE("Reading segment data from page to server response event"
+                               COPY_TRACE(event->data, mapped_page,
+                                       EVENT_BLOCK_SIZE));
+               memcpy(event->data, mapped_page, EVENT_BLOCK_SIZE);
+
+//             print_hex_dump(KERN_INFO, "server event", DUMP_PREFIX_OFFSET,
+//                     16, 1, event->data, EVENT_BLOCK_SIZE, true);
+//             print_hex_dump(KERN_INFO, "server page", DUMP_PREFIX_OFFSET,
+//                     16, 1, mapped_page, EVENT_BLOCK_SIZE, true);
+
+               kunmap_atomic(mapped_page);
+               mempool_free(bvec->bv_page, dev->page_pool);
+               bvec->bv_page = NULL;
+       }
+
+       bio->bi_vcnt = 0;
+
+       cleanup_bio(bio);
+
+       return error;
+}
+
+static int muenblk_server_write_resp(struct muenblk_device *dev,
+       struct muenblk_event *event, struct bio *bio)
+{
+       struct bio_vec *bvec = NULL;
+       int i = 0;
+       int error = 0;
+
+       TRACE("Processing WRITE response" BIO_TRACE(bio));
+
+       event->type = MUENBLK_EVENT_WRITE;
+       event->id = ((bio->bi_iter.bi_sector -
+                               dev->disk_kernel_start_sector) >> 3) - 1;
+
+       bio_for_each_segment_all(bvec, bio, i) {
+               mempool_free(bvec->bv_page, dev->page_pool);
+               bvec->bv_page = NULL;
+       }
+
+       bio->bi_vcnt = 0;
+
+       cleanup_bio(bio);
+
+       return error;
+}
+
+static int muenblk_server_command_resp(struct muenblk_device *dev,
+       struct muenblk_event *event, struct bio *bio)
+{
+       int error = 0;
+
+       TRACE("Processing FLUSH COMMAND command response" BIO_TRACE(bio));
+
+       event->type = MUENBLK_EVENT_COMMAND;
+       event->id = MUENBLK_COMMAND_SYNC;
+
+       cleanup_bio(bio);
+
+       return error;
+}
+
+static int muenblk_server_dummy_command_resp(struct muenblk_device *dev,
+       struct muenblk_event *event, struct bio *bio)
+{
+       if (bio == dev->dummy.size_bio) {
+               TRACE("Processing SIZE COMMAND response" BIO_TRACE(bio));
+
+               event->type = MUENBLK_EVENT_COMMAND;
+               event->id = MUENBLK_COMMAND_SIZE;
+               memcpy(event->data, &dev->disk_sector_count,
+                       sizeof(dev->disk_sector_count));
+       }
+
+       return 0;
+}
+
+static int muenblk_server_transfer_resp(struct muenblk_device *dev,
+       struct muenblk_event *event, struct bio *bio)
+{
+       struct muenblk_priv_data *metadata = NULL;
+       int error = 0;
+
+       metadata = bio->bi_private;
+
+       event->priv = metadata->priv;
+       event->error = metadata->error;
+       memset(event->data, 0, EVENT_BLOCK_SIZE);
+
+       if (unlikely(muenblk_is_dummy_bio(dev, bio))) {
+               error = muenblk_server_dummy_command_resp(dev, event, bio);
+       } else {
+               switch (bio_op(bio)) {
+               case REQ_OP_FLUSH:
+                       error = muenblk_server_command_resp(dev, event, bio);
+                       break;
+               case REQ_OP_WRITE:
+                       error = muenblk_server_write_resp(dev, event, bio);
+                       break;
+               case REQ_OP_READ:
+                       error = muenblk_server_read_resp(dev, event, bio);
+                       break;
+               default:
+                       TRACE("Unexpected op" BIO_TRACE(bio));
+                       BUG_ON(bio->bi_opf);
+               }
+       }
+
+       muenblk_server_send_resp(dev, event);
+
+       return error;
+}
+
+static void muenblk_server_queue_resp(struct muenblk_device *dev,
+       struct bio *bio)
+{
+       unsigned long flags = 0;
+
+       TRACE("Adding response into response queue" BIO_TRACE(bio));
+
+       spin_lock_irqsave(&dev->resp.lock, flags);
+
+       bio_list_add(&dev->resp.bio_list, bio);
+
+       atomic_inc(&dev->resp.in_flight);
+
+       spin_unlock_irqrestore(&dev->resp.lock, flags);
+
+       wake_up_interruptible(&dev->resp.wq);
+}
+
+static void muenblk_server_cb_endio(struct bio *bio)
+{
+       struct muenblk_priv_data *metadata;
+
+       BUG_ON(!bio);
+
+       metadata = bio->bi_private;
+       metadata->error = bio->bi_error;
+
+       if (unlikely(bio->bi_error))
+               ERROR("BIO EndIO invoked with error: %d", bio->bi_error);
+
+       TRACE("BIO EndIO invoked" BIO_TRACE(bio));
+
+       muenblk_server_queue_resp(metadata->dev, bio);
+}
+
+static int muenblk_server_req_thread(void *data)
+{
+       struct muenblk_device *dev = data;
+
+       TRACE("Entering RequestThread\n");
+       muenblk_read_packets(dev, &dev->req, &muenblk_server_transfer_req,
+                       true);
+       TRACE("Leaving RequestThread\n");
+       return 0;
+}
+
+static int muenblk_server_resp_thread(void *data)
+{
+       struct muenblk_device *dev = data;
+       struct bio *bio = NULL;
+       unsigned long flags = 0;
+       int error = 0;
+
+       TRACE("Entering ResponseThread\n");
+
+       while (!kthread_should_stop()) {
+               error = wait_event_interruptible(dev->resp.wq,
+                       atomic_read(&dev->resp.in_flight));
+
+               if (unlikely(kthread_should_stop())) {
+                       TRACE("ResponseThread got STOP SIGNAL -> Terminating\n");
+                       goto out_thread;
+               }
+
+               if (unlikely(error == -ERESTARTSYS || error))
+                       continue;
+
+               spin_lock_irqsave(&dev->resp.lock, flags);
+
+               bio = bio_list_pop(&dev->resp.bio_list);
+               atomic_dec(&dev->resp.in_flight);
+
+               spin_unlock_irqrestore(&dev->resp.lock, flags);
+
+               BUG_ON(!bio);
+
+               TRACE("ResponseThread processing response" BIO_TRACE(bio));
+
+               error = muenblk_server_transfer_resp(dev, dev->resp.event, bio);
+
+               if (unlikely(error))
+                       ERROR("ResponseThread processing response with error(s) %d\n", error);
+       }
+
+out_thread:
+       TRACE("Leaving ResponseThread\n");
+
+       return 0;
+}
+
+static int muenblk_server_init_block_device(struct muenblk_device *dev)
+{
+       int error = 0;
+       fmode_t mode = FMODE_READ | FMODE_WRITE | FMODE_LSEEK;
+
+       struct block_device *bdev;
+
+       bdev = blkdev_get_by_path(dev->disk_name, mode, dev);
+       if (IS_ERR(bdev)) {
+               ERROR("Failed to retrieve selected block device '%s'\n", dev->disk_name);
+               error = PTR_ERR(bdev);
+               goto exit;
+       }
+
+       dev->bdev = bdev;
+       dev->disk = bdev->bd_disk;
+       dev->mode = mode;
+
+       get_disk(bdev->bd_disk);
+
+       dev->disk_kernel_sector_count = bdev->bd_part->nr_sects;
+       dev->disk_kernel_start_sector = bdev->bd_part->start_sect;
+       dev->disk_sector_count = dev->disk_kernel_sector_count;
+       dev->disk_size = dev->disk_kernel_sector_count * KERNEL_SECTOR_SIZE;
+
+       dev->queue = NULL;
+
+       INFO("Successfully initialized block device %s (%llu, %llu, %llu, %llu)\n",
+               dev->disk_name,
+               dev->disk_size,
+               dev->disk_sector_count,
+               dev->disk_kernel_sector_count,
+               dev->disk_kernel_start_sector);
+
+exit:
+       return error;
+}
+
+static int muenblk_server_init_mem(struct muenblk_device *dev)
+{
+       int error = 0;
+       int num = 0;
+
+       //XXX: Why +2?
+       num = dev->req.channel_queue_size + dev->resp.channel_queue_size + 2;
+
+       dev->bio_set = bioset_create(num, 0);
+       if (!dev->bio_set) {
+               ERROR("Failed to create bioset\n");
+
+               error = -ENOMEM;
+               goto exit;
+       }
+
+       dev->page_pool = mempool_create_page_pool(num, 0);
+       if (!dev->page_pool) {
+               ERROR("Failed to create physical page pool\n");
+
+               error = -ENOMEM;
+               goto err_cleanup_bioset;
+       }
+
+       dev->metadata_pool = mempool_create_kmalloc_pool(num,
+               sizeof(struct muenblk_priv_data));
+       if (!dev->metadata_pool) {
+               ERROR("Failed to create metadata pool\n");
+
+               error = -ENOMEM;
+               goto err_cleanup_pagepool;
+       }
+
+       TRACE("Successfully initialized server memory\n");
+
+       return 0;
+
+err_cleanup_pagepool:
+       mempool_destroy(dev->page_pool);
+       dev->page_pool = NULL;
+
+err_cleanup_bioset:
+       bioset_free(dev->bio_set);
+       dev->bio_set = NULL;
+
+exit:
+       return error;
+}
+
+static void muenblk_server_cleanup_mem(struct muenblk_device *dev)
+{
+       if (dev->bio_set) {
+               bioset_free(dev->bio_set);
+               dev->bio_set = NULL;
+       }
+
+       mempool_destroy(dev->page_pool);
+       dev->page_pool = NULL;
+
+       mempool_destroy(dev->metadata_pool);
+       dev->metadata_pool = NULL;
+
+       TRACE("Server memory cleanup successful\n");
+}
+
+static void muenblk_server_cleanup_block_device(struct muenblk_device *dev)
+{
+       put_disk(dev->disk);
+       blkdev_put(dev->bdev, dev->mode);
+
+       dev->disk = NULL;
+       dev->bdev = NULL;
+
+       TRACE("Block device cleanup successful\n");
+}
+
+static void muenblk_server_exit_dev(struct muenblk_device *dev)
+{
+       if (dev) {
+               muenblk_cleanup_shm(dev, true);
+               muenblk_server_cleanup_mem(dev);
+               muenblk_cleanup_dummy_bio(dev);
+               muenblk_server_cleanup_block_device(dev);
+
+               kfree(dev);
+       }
+}
+
+static void muenblk_server_exit(void)
+{
+       int i = 0;
+
+       for (i = 0; i < dev_name_count; i++) {
+               muenblk_server_exit_dev(blk_dev[i]);
+               blk_dev[i] = NULL;
+       }
+}
+
+static int __init muenblk_server_init(void)
+{
+       int i = 0;
+       unsigned int len = 0;
+       int error = 0;
+
+       struct muenblk_device *dev = NULL;
+
+       if (!dev_name_count)
+               return -EINVAL;
+
+       for (i = 0; i < dev_name_count; i++) {
+               dev = kzalloc(sizeof(struct muenblk_device), GFP_KERNEL);
+
+               if (!dev) {
+                       error = -ENOMEM;
+                       goto err_out;
+               }
+
+               mutex_init(&dev->mutex);
+               init_waitqueue_head(&dev->wq);
+               init_waitqueue_head(&dev->flow_wq);
+
+               dev->req.thread_fn = muenblk_server_req_thread;
+               dev->resp.thread_fn = muenblk_server_resp_thread;
+               dev->poll_interval = poll_interval;
+               dev->disk_major = 0;
+               dev->disk_minor = 0;
+               dev->disk_index = i;
+
+               len = strlen(device_name[i]);
+               if (len >= DISK_NAME_LEN)
+                       goto err_dev_cleanup;
+
+               strncpy(dev->disk_name, device_name[i], len);
+               dev->disk_name[len] = 0;
+
+               len = strlen(req_shm_name[i]);
+               if (len >= 64)
+                       goto err_dev_cleanup;
+
+               strncpy(dev->req.channel_name, req_shm_name[i], len);
+               dev->req.channel_name[len] = 0;
+
+               len = strlen(resp_shm_name[i]);
+               if (len >= 64)
+                       goto err_dev_cleanup;
+
+               strncpy(dev->resp.channel_name, resp_shm_name[i], len);
+               dev->resp.channel_name[len] = 0;
+
+               if (req_shm_protocol[i] && strlen(req_shm_protocol[i]) > 0) {
+                       error = kstrtoull(req_shm_protocol[i], 16,
+                                       &dev->req.channel_protocol);
+                       if (error) {
+                               ERROR("Invalid request protocol '%s'\n",
+                                               req_shm_protocol[i]);
+                               goto err_dev_cleanup;
+                       }
+               } else
+                       dev->req.channel_protocol = 0;
+
+               if (resp_shm_protocol[i] && strlen(resp_shm_protocol[i]) > 0) {
+                       error = kstrtoull(resp_shm_protocol[i], 16,
+                                       &dev->resp.channel_protocol);
+                       if (error) {
+                               ERROR("Invalid response protocol '%s'\n",
+                                               req_shm_protocol[i]);
+                               goto err_dev_cleanup;
+                       }
+               } else
+                       dev->resp.channel_protocol = 0;
+
+               error = muenblk_init_dummy_bio(dev);
+               if (error)
+                       goto err_dev_cleanup;
+
+               error = muenblk_server_init_block_device(dev);
+               if (error)
+                       goto err_dummy_cleanup;
+
+               error = muenblk_init_shm(dev, true);
+               if (error)
+                       goto err_bdev_cleanup;
+
+               error = muenblk_server_init_mem(dev);
+               if (error)
+                       goto err_shm_cleanup;
+
+               blk_dev[i] = dev;
+
+               wake_up_interruptible(&dev->req.wq);
+
+               if (error)
+                       goto err_mem_cleanup;
+
+       }
+
+       TRACE("Successfully initialized module\n");
+
+       return 0;
+
+err_mem_cleanup:
+       muenblk_server_cleanup_mem(dev);
+
+err_shm_cleanup:
+       muenblk_cleanup_shm(dev, true);
+
+err_bdev_cleanup:
+       muenblk_server_cleanup_block_device(dev);
+
+err_dummy_cleanup:
+       muenblk_cleanup_dummy_bio(dev);
+
+err_dev_cleanup:
+       kfree(dev);
+
+err_out:
+       muenblk_server_exit();
+
+       return error;
+}
+
+module_init(muenblk_server_init);
+module_exit(muenblk_server_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Dennis Wassenberg <dennis.wassenberg@secunet.com>");
+MODULE_DESCRIPTION("Muen SK block device driver (server)");
+
+module_param_array(req_shm_name, charp, NULL, 0444);
+MODULE_PARM_DESC(req_shm_name, "List of request shared memory region names.");
+
+module_param_array(resp_shm_name, charp, NULL, 0444);
+MODULE_PARM_DESC(resp_shm_name, "List of response shared memory region names.");
+
+module_param_array(req_shm_protocol, charp, NULL, 0444);
+MODULE_PARM_DESC(req_shm_protocol, "List of request shared memory protocols.");
+
+module_param_array(resp_shm_protocol, charp, NULL, 0444);
+MODULE_PARM_DESC(resp_shm_protocol, "List of response shared memory protocol.");
+
+module_param_array(device_name, charp, &dev_name_count, 0444);
+MODULE_PARM_DESC(device_name, "List of names of block devices to use.");
+
+module_param(poll_interval, uint, 0444);
+MODULE_PARM_DESC(poll_interval, "Polling interval.");