ZFS Storage Appliance Infiniband
To attach your ZFSSA using infiniband to an EXA for example you might want more than one "virtual" datalink on your HCA using multiple IB partitions with same pkey. It is (yet?) not possible to allow the use of the same pkey when creating two IB partitions (datalinks) that point to the same IB device using the BUI and the CLI property is hidden; don't know why; but I could find a workaround in: Oracle Support Document 2087231.1 (Guidelines When Using ZFS Storage in an Exadata Environment) https://support.oracle.com/epmos/faces/DocumentDisplay?id=2087231.1
In my example I will create a bunch of datalinks to enable an active-active IPMP failover configuration for both controllers. (you will have to start with ibpart1, "0" does not work)
zsexa0101a:configuration net datalinks>partition
zsexa0101a:configuration net datalinks partition (uncommitted)> set li <-- tab tab
linkmode links
zsexa0101a:configuration net datalinks partition (uncommitted)> set link <-- tab tab
linkmode links <-- it is not there ;-|
zsexa0101a:configuration net datalinks partition (uncommitted)> set linkname=ibpart5
linkname = ibpart5 (uncommitted)
zsexa0101a:configuration net datalinks partition (uncommitted)> show
Properties:
class = partition
label = Untitled Datalink
links = (unset)
pkey = (unset)
linkmode = cm
zsexa0101a:configuration net datalinks partition (uncommitted)> set links=ibp0
links = ibp0 (uncommitted)
zsexa0101a:configuration net datalinks partition (uncommitted)> set pkey=ffff
pkey = ffff (uncommitted)
zsexa0101a:configuration net datalinks partition (uncommitted)> show
Properties:
class = partition
label = Untitled Datalink
links = ibp0 (uncommitted)
pkey = ffff (uncommitted)
linkmode = cm
zsexa0101a:configuration net datalinks partition (uncommitted)> commit
zsexa0101a:configuration net datalinks> show
Datalinks:
DATALINK CLASS LINKS STATE ID LABEL
aggr1 aggregation i40e0 up - zsexa01-LACP
i40e4
ibpart1 partition ibp0 up - zsexa01-IB0
ibpart2 partition ibp2 up - zsexa01-IB1
ibpart3 partition ibp0 up - zsexa01-IB0
ibpart4 partition ibp2 up - zsexa01-IB1
ibpart5 partition ibp0 up - Untitled Datalink
igb0 device igb0 up - Motherboard-igb0
pffff_ibp0 partition ibp0 up - zsexa01-IB0
pffff_ibp2 partition ibp2 up - zsexa01-IB1
vnic1 vnic igb0 up - zsexa0101a-VNIC
vnic2 vnic igb0 up - zsexa0102a-VNIC
vnic3 vnic aggr1 up - zsexa0101c-VNIC
vnic4 vnic aggr1 up - zsexa0102c-VNIC