Although the canonical source of reference for QMP commands is
qapi-schema.json, for consistency's sake, update qmp-commands.hx to
state the list of supported transactionable operations, namely:

    drive-backup
    blockdev-backup
    blockdev-snapshot-internal-sync
    abort
    block-dirty-bitmap-add
    block-dirty-bitmap-clear

Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>

Use a transaction to request an incremental backup across two drives.
Coerce one of the jobs to fail, and then re-run the transaction.

Verify that no bitmap data was lost due to the partial transaction
failure.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>

This patch actually implements the transactional callback system
for the drive_backup action.

(1) We create a functional closure that envelops the original drive_backup
    callback, to be able to intercept the completion status and return code
    for the job.

(2) We inform the BlockJob layer that this job is involved in a transaction,
    which just increments a reference on the bitmap.

(3) We add the drive_backup_cb method for the drive_backup action, which
    unpacks the completion information and invokes the final cleanup.

(4) backup_action_complete will perform the final cleanup on the
    opaque object (a BdrvDirtyBitmap, here) returned earlier.

(5) In the case of transaction cancellation, drive_backup_cb is still
    responsible for cleaning up the mess we may have already made.

Signed-off-by: John Snow <jsnow@redhat.com>

We'd like to be able to specify the callback given to backup_start
manually in the case of transactions, so split apart qmp_drive_backup
into an implementation and a wrapper.

Switch drive_backup_prepare to use the new wrapper, but don't overload
the callback and closure yet.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>

Allow bitmap successors to carry reference counts.

We can in a later patch use this ability to clean up the dirty bitmap
according to both the individual job's success and the success of all
jobs in the transaction group.

As a consequence of moving the bitmap successor cleanup actions behind
the frozen bitmap decref operation, the previous operations are made
static, their errp parameter deleted, and some user-facing errors are
replaced by assertions.

"Also," The code for cleaning up a bitmap is also moved from backup_run to
backup_complete.

Signed-off-by: John Snow <jsnow@redhat.com>

The goal here is to add a new method to transactions that allows
developers to specify a callback that will get invoked only once
all jobs spawned by a transaction are completed, allowing developers
the chance to perform actions conditionally pending complete success,
partial failure, or complete failure.

In order to register the new callback to be invoked, a user must request
a callback pointer and closure by calling new_action_cb_wrapper, which
creates a wrapper around an opaque pointer and callback that would have
originally been passed to e.g. backup_start().

The function will return a function pointer and a new opaque pointer to
be passed instead. The transaction system will effectively intercept the
original callbacks and perform book-keeping on the transaction after it
has delivered the original enveloped callback.

This means that Transaction Action callback methods will be called after
all callbacks triggered by all Actions in the Transactional group have
been received.

This feature has no knowledge of any jobs spawned by Actions that do not
inform the system via new_action_cb_wrapper().

For an example of how to use the feature, please skip ahead to:
'block: drive_backup transaction callback support' which serves as an example
for how to hook up a post-transaction callback to the Drive Backup action.


Note 1: Defining a callback method alone is not sufficient to have the new
        method invoked. You must call new_action_cb_wrapper() AND ensure the
        callback it returns is the one used as the callback for the job
        launched by the action.

Note 2: You can use this feature for any system that registers completions of
        an asynchronous task via a callback of the form
        (void *opaque, int ret), not just block job callbacks.

Signed-off-by: John Snow <jsnow@redhat.com>

Now that the structure formerly known as BlkTransactionState has been
renamed to something sensible (BlkActionState), re-introduce an actual
BlkTransactionState that actually manages state for the entire Transaction.

In the process, convert the old QSIMPLEQ list of actions into a QTAILQ,
to let us more efficiently delete items in arbitrary order, which will
be more important in the future when some actions will expire at the end
of the transaction, but others may persist until all callbacks triggered
by the transaction are recollected.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>

These structures are misnomers, somewhat.

(1) BlockTransactionState is not state for a transaction,
    but is rather state for a single transaction action.
    Rename it "BlkActionState" to be more accurate.

(2) The BdrvActionOps describes operations for the BlkActionState,
    above. This name might imply a 'BdrvAction' or a 'BdrvActionState',
    which there isn't.
    Rename this to 'BlkActionOps' to match 'BlkActionState'.

Lastly, update the surrounding in-line documentation and comments
to reflect the current nature of how Transactions operate.

This patch changes only comments and names, and should not affect
behavior in any way.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

Test simple usage cases for using transactions to create
and synchronize incremental backups.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

This adds two qmp commands to transactions.

block-dirty-bitmap-add allows you to create a bitmap simultaneously
alongside a new full backup to accomplish a clean synchronization
point.

block-dirty-bitmap-clear allows you to reset a bitmap back to as-if
it were new, which can also be used alongside a full backup to
accomplish a clean synchronization point.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

…ream' into staging

Patch queue for s390 - 2015-06-05

This time there are a lot of s390x TCG emulation bug fixes - almost all
of them from Aurelien, who returned from nirvana :).

# gpg: Signature made Fri Jun  5 00:39:27 2015 BST using RSA key ID 03FEDC60
# gpg: Good signature from "Alexander Graf <agraf@suse.de>"
# gpg:                 aka "Alexander Graf <alex@csgraf.de>"

* remotes/agraf/tags/signed-s390-for-upstream: (34 commits)
  target-s390x: Only access allocated storage keys
  target-s390x: fix MVC instruction when areas overlap
  target-s390x: use softmmu functions for mvcp/mvcs
  target-s390x: support non current ASC in s390_cpu_handle_mmu_fault
  target-s390x: add a cpu_mmu_idx_to_asc function
  target-s390x: implement high-word facility
  target-s390x: implement load-and-trap facility
  target-s390x: implement miscellaneous-instruction-extensions facility
  target-s390x: implement LPDFR and LNDFR instructions
  target-s390x: implement TRANSLATE EXTENDED instruction
  target-s390x: implement TRANSLATE AND TEST instruction
  target-s390x: implement LOAD FP INTEGER instructions
  target-s390x: move SET DFP ROUNDING MODE to the correct facility
  target-s390x: move STORE CLOCK FAST to the correct facility
  target-s390x: change CHRL and CGHRL format to RIL-b
  target-s390x: fix CLGIT instruction
  target-s390x: fix exception for invalid operation code
  target-s390x: implement LAY and LAEY instructions
  target-s390x: move a few instructions to the correct facility
  target-s390x: detect tininess before rounding for FP operations
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

We allocate ram_size / PAGE_SIZE storage keys, so we need to make sure that
we only access that many. Unfortunately the code can overrun this array by
one, potentially overwriting unrelated memory.

Fix it by limiting storage keys to their scope.

Signed-off-by: Alexander Graf <agraf@suse.de>
Reviewed-by: Aurelien Jarno <aurelien@aurel32.net>

The MVC instruction and the memmove C funtion do not have the same
semantic when memory areas overlap:

MVC: When the operands overlap, the result is obtained as if the
operands were processed one byte at a time and each result byte were
stored immediately after fetching the necessary operand byte.

memmove: Copying takes place as though the bytes in src are first copied
into a temporary array that does not overlap src or dest, and the bytes
are then copied from the temporary array to dest.

The behaviour is therefore the same when the destination is at a lower
address than the source, but not in the other case. This is actually a
trick for propagating a value to an area. While the current code detects
that and call memset in that case, it only does for 1-byte value. This
trick can and is used for propagating two or more bytes to an area.

In the softmmu case, the call to mvc_fast_memmove is correct as the
above tests verify that source and destination are each within a page,
and both in a different page. The part doing the move 8 bytes by 8 bytes
is wrong and we need to check that if the source and destination
overlap, they do with a distance of minimum 8 bytes before copying 8
bytes at a time.

In the user code, we should check check that the destination is at a
lower address than source or than the end of the source is at a lower
address than the destination before calling memmove. In the opposite
case we fallback to the same code as the softmmu one. Note that l
represents (length - 1).

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

mvcp and mvcs helper get access to the physical memory by a call to
mmu_translate for the virtual to real conversion and then using ldb_phys
and stb_phys to physically access the data. In practice this is quite
slow because it bypasses the QEMU softmmu TLB and because stb_phys calls
try to invalidate the corresponding memory for each access.

Instead use cpu_ldb_{primary,secondary} for the loads and
cpu_stb_{primary,secondary} for the stores. Ideally this should be
further optimized by a call to memcpy, but that already improves the
boot time of a guest by a factor 1.8.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

s390_cpu_handle_mmu_fault currently looks at the current ASC mode
defined in PSW mask instead of the MMU index. This prevent emulating
easily instructions using a specific ASC mode. Fix that by using the
MMU index converted back to ASC using the just added cpu_mmu_idx_to_asc
function.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

Use constants to define the MMU indexes, and add a function to do
the reverse conversion of cpu_mmu_index.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

Besides RISBHG and RISBLG, all high-word instructions are not
implemented. Fix that.

Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

At the same time move the trap code from op_ct into gen_trap and use it
for all new functions. The value needs to be stored back to register
before the exception, but also before the brcond (as we don't use
temp locals). That's why we can't use wout helper.

Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

RISBGN is the same as RISBG, but without setting the condition code.
CLT and CLGT are the same as CLRT and CLGRT, but using memory for the
second operand.

Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

This complete the floating point support sign handling facility.

Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

It is part of the basic zArchitecture instructions.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

It is part of the basic zArchitecture instructions. Allow it to be call
from EXECUTE.

Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

This is needed to pass the gcc.c-torture/execute/ieee/20010114-2.c test
in the gcc testsuite.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

It belongs to the DFP rounding facility.

Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

STORE CLOCK FAST should be in the SCF facility.

Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

Change to match the PoP. In practice both format RIL-a and RIL-b have
the same fields. They differ on the way we decode the fields, and it's
done correctly in QEMU.

Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

The COMPARE LOGICAL IMMEDIATE AND TRAP instruction should compare the
numbers as unsigned, as its name implies.

Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

When an operation code is not recognized (ie invalid instruction) an
operation exception should be generated instead of a specification
exception. The latter is for valid opcode, with invalid operands or
modifiers.

This give a very basic GDB support in the guest, as it uses the invalid
opcode 0x0001 to generate a trap.

Reviewed-by: Richard Henderson <rth@twiddle.net>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

This complete the general-instructions-extension facility, enable it.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
[agraf: remove facility bit]
Signed-off-by: Alexander Graf <agraf@suse.de>

LY is part of the long-displacement facility.
RISBHG and RISBLG are part of the high-word facility.
STCMH is part of the z/Architecture.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

The s390x floating point unit detects tininess before rounding, so set
the softfloat fp_status up appropriately.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

LOAD LENGTHENED and LOAD ROUNDED are considered as FP operations and
thus need to convert input sNaN into corresponding qNaN.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

The cpu_mmu_index function wrongly looks at PSW P bit to determine the
MMU index, while this bit actually only control the use of priviledge
instructions. The addressing mode is detected by looking at the PSW ASC
bits instead.

This used to work more or less correctly up to kernel 3.6 as the kernel
was running in primary space and userland in secondary space. Since
kernel 3.7 the default is to run the kernel in home space and userland
in primary space. While the current QEMU code seems to work it open some
security issues, like accessing the lowcore memory in R/W mode from a
userspace process once it has been accessed by the kernel (it is then
cached by the QEMU TLB).

At the same time change the MMU_USER_IDX value so that it matches the
value used in recent kernels.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>

runtime_exception computes the psw.addr value using the actual exception
address and the instruction length computed by calling the get_ilen
function. However as explained above the get_ilen code, it returns the
actual instruction length, and not the ILC. Therefore there is no need to
multiply the value by 2.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alexander Graf <agraf@suse.de>