Fibre Channel Notes from Learning Session

San Components:  Multi-Protocol Storage Routers  (ex. MDS 9222i)

  • Supports FC, FCIP, iSCSI (Initiators and Targets)
  • 1,2,4,8 and 10 Gbps FC and 1Gbps Ethernet (10 G is for Switch to Switch trunking.  Not HBA’s exist)
  • Server Based config and management just like Nexus with DCNM

Nexus 5000

  • Combined Ethernet and FC switch
  • 8 ports of 1/2/4 FC or 6 ports of 2/4/8 FC
  • Requires both Fabric Manager and CLI or DCNM

Why Build a SAN

  • Improved flexibility and non-disruptive service scalability

Fiber Channel:  Flow Based vs Collision based control

  • High Speed
  • High Avail
  • Block Data Based
  • Peer to Peer
  • Hardware Intensive
  • %95 of FC uses SCSI

Class of Service:  Most is Class 3

  • Class1:  Connect oriented and confirmed delivery
  • Class2:  Packet Switched and confirmed delivery
  • Class3:  Packet switched Non confirm delivery
  • Class4:  Fractional bandwidth virtual circuit and confirmed delivery
  • Class6:  Connection orientiented  multicast with confirmed delivery
  • ClassF:  Packet Switch Confirmed  (Switch to Switch Coms)  Auto defined in Fabric Build
  • 40 Nodes in a Fabric max  (Fabric nodes = switches) ?

Topologies:

  • Point to Point
  • Arbitrated Loop (similar to FDDI)
  • Switched fabric  (Star)  Mixed speed possible

Switched Topology:

Max nodes = 16 million

Idles are “ordered sets”  used for sync and signaling

Frame Length = 2148 Bytes

FC Port Types:

N Port:  Node port used to connect to switch fabric or point to point (Server or Storage PortP

F Port:  Fabric ports on switches for connecting to N port devices.

xL Port:  Loop ports used in arbitrated loop config.  Seldom used

E Port:  Expansion Ports used to connet two FC switches

Auto Port:  Generic port able to auto negotiate E or F:  Auto Discovery

Port Types can be used for low level security design.

Addition Cisco Port Types:

SD:  Span

TL:  Translative Loop  ( No Longer seen)

ST:  Span Tunnel,

TE:  Trunking mode  functions as an expansion port  can be connected to another TE to created extended ISL between swiches only found in Cisco MDS 9000 switches

NP:  Proxy N Port

Inter-Switch LINK  (ISL)

Switch to Switch = Eport to Eport

Supports all Class of Services

Fiber Channel is a Flow based Control using Buffer to Buffer Credits

These are defined on a link to link basis

Fiber Channel World Wide Name (WWN)

Similar to Mac But longer and always unique

  • Each Switch is unique
  • Each Port is unique
  • Each host is unique

FC-IS is assigned by FC switch during the Fabric Login (FLOGI)

Unique Domain number (8bits) and is assigned by Principal Switch  (Statically or Dynamically designed)

UD numbers are 1 – 239 ( Each switch = a new domain, in VSANS  a switch can be multiple UDS

Max Switch domains = 239, max supported = 40 @ 239 most traffic would be class F and not real storage traffic.  Cisco can support up to 75.

Typical FC design = A fab and B fab so always redundant

If you change modes of operation it is disruptive.  Cisco can do interop mode or brocade mode or NPV (endpoint virtualization mode, becomes transparent to brocade and the 5000 becomes a pass-through device and is a global config setting and requires NPIV functionality in brocade is non-disruptive)

UCS Extender is NPV Mode only

Each switch has its own FLOGI Services (Login Server, Name Server)  Class F traffic is used to share this info  (admin traffic)

Virtual SAN (VSAN) MDS 9000 Family (equiv of VLAN in Storage)

Each port lives in a vsan

Up to 16/256 VSANS in a single Switch/Director

Logical config to move a port from one fabric to another

WWN-Based VSANS can provide auto VSAN Membership

Each VSAN has its own Services and are full Virtual fabrics

Inter VSAN Routing = more lic and more config

Fabric disruptions are limited to VSANS

SAN Zoning:

Zoning arrangeds FC connected devices int logical groups and allows lun policies to control access to luns from zones

Zone members can be part of multiple zones unlike VSANS.  These should be seen as storage Access Lists.  Zonesets are per VSAN

Default Zone should be disabled.  Otherwise no access policy can be applied.  Plus admin traffic dramatically increases

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.