============================
R E L E A S E    N O T E S
============================

QLogic qfle3 VMware ESX Native Driver for ESXi 6.7
Copyright (c) 2018 QLogic Corporation
All rights reserved

ESXi 6.7 : Version 1.0.69.0 (April 16, 2018) Internal FW: 7.13.109.0
Fixes:
----------
1. Cont00100086 : UMCE observed after upgrading from BNX2X to QLFE3
   Change: Disable stats collection from adapter during the execution of 
           reboot handler. 
2. Prevent calling of of MF BW configuration when we are in single function 
mode.

ESXi 6.7 : Version 1.0.68.0 (March 21, 2018) Internal FW: 7.13.109.0
Fixes:
----------
1. Cont00100480  : Response timeout seen in VM at time of VF probe request.
   Changes:  Fix VF sp_cl_id.

ESXi 6.7 : Version 1.0.67.0 (March 12, 2018) Internal FW: 7.13.109.0
Fixes:
----------
1. Cont00098970 : Unable to unload qfle3 module, when vxlan_filter is enabled.
   Changes: There were two different variables defined in 'bxe_sp_objs' data
	structure for vxlan object, which was preventing one spinlock of cnic fastpath 
	from being destroyed. Removed one vxlan object unused vaiabled from 'bxe_sp_objs',
	which solves the issue. 
   
Enhancement:
----------
1. Modified Default Verbosity of driver Information Messages. 

2. Add module parameter to configure the fairness threshold values.


ESXi 6.7 : Version 1.0.66.0 (March 06,2018) Internal FW: 7.13.3.0
Fixes:
----------
1. Cont00099967: Correct "rxqueue_nr" and "txqueue_nr" driver module
   parameter info.
   Changes:      Updated driver module parameter to print correct number of
                 maximum supported tx and rx queues.
2. Cont00099998: Incorrect calculation of multirss engines when changing the RX queue using 
   driver module parameters.
   Changes:      Add check to reduce max. supported RSS Engines equal or less than
                 max. configured RX queues using driver module parameter.
3. Cont00100120: E3-qfle3-ESX6.7-NPAR: Getting PSOD after enabling NPAR boot to ESX
   Changes:      Fixed regression introduced due to inbox changes.
4. Cont00099993: Multi-RSS feature in not functioning for NIC partitions other then 
                 base functions.
   Changes:      Fixed regression introduced due to inbox changes.
5. Cont00099709: PSOD while running DDV EILoading IOVP test case for NIC 6.7.
   Changes: 	 Handle the errors properly.
6. Cont00099934: PSOD while running DeviceStateWithGracefulRemove test case
                 for NIC 6.7 OS.
   Changes:	 Change made for Cont00099709 addresses this issue as well.
7. Cont00100084: E3 Driver qfle3 is unable to create the new filters and queue after 
                 continuously doing the NIC up down.
   Changes:      Regression introduced due to inbox changes: Filter delete was not executed on 
                 correct Primary RSS Queue during qfle3_rq_stop execution.  

ESXi 6.7 : Version 1.0.65.0 (Feb 21,2018) Internal FW: 7.13.3.0
Fixes:
----------
1. This version contains all the fixes made to the inbox driver for vsphere 
2018 that was not already ported to async driver.  

Enhancement:
------------
1. Software FCoE support.  Introducing support for multi-cos TX queues.
2. Fast reboot support.

ESXi 6.7 : Version 1.0.64.0 (Feb 7,2018) Internal FW: 7.13.3.0
Fixes:
----------
1. Cont00099203: E3-MultiRss not able to get secondary queues info
   Changes: Execute "vmk_UplinkQueueMkRxQueueID" for Sec Queues
2. Cont00099202: E3-MultiRss after starting traffic logs are flooded 
                 with return busy packets.
   Changes: Fixing Sec Queue ID fixed this issue.

ESXi 6.5 : Version 1.0.63.0 (Jan 30,2018) Internal FW: 7.13.3.0
Fixes:
----------
1.  Cont00098859: qfle3 module show busy while unload
    Changes:  do not return error when detach is called while the link is down.
2.  Cont00099288: Error installing cna-bundle : cna-bundle-1.0.53.tgz
    Changes: Modify qfle3.h file to get ESX6.7 qfle3 work with ESX6.5 
             qcnic driver. 

ESXi 6.5 : Version 1.0.61.0 (Jan 18,2018) Internal FW: 7.13.3.0
Enhancement:
------------
1. Multi Netqueue RSS.
Fixes:
----------
1. Modification in DRSS code to let DRSS get enabled on first Functions when 
adapter is configured
2. Cont00097611:- NPAR:PSOD during reboot. 
	Changes:- vmk_RegisterRebootHandler() moved it to init_module() from attach() 
	and vmk_UnregisterRebootHandler() moved it to cleanup_module() from detach(). 
	So that it will get called per-module instead of per-adapter. 
3. Cont00090708: ESXi65 PSOD with "qfle3_stats_update_fuct" error in NPAR mode. 
4. Fixed a problem reported by vmware where the queues don't get restarted 
after uplink reset.
   Changes:  change the statemachine to update the SQD state variable and also 
   send vmk_UplinkQueueStop/vmk_UplinkQueueStart calls.
5. Cont00098000 :- Family Driver version in server console showing as "Unavailable".
   Changes :- Implemented qfle3_update_mng_version() to display correct driver
		version.
6. Cont00098587 - ESXi6.5U1 rss queue fail to start.
   Change:- When driver doesn't have enough resources to support DRSS queue
            when set by user, driver will recalculate the rx and tx queues by disabling DRSS.

7. Cont00097673 - Unable to collect the SRIOV stats using command <esxcli network sriov nic >
   Change: Added support for "VMK_NETPTOP_VF_GET_QUEUE_STATS" in qfle3 driver.

8. Put back CID leak fix
   Change: Set the TM_REG_EN_LINEAR0_TIMER time scan when CNIC queues are 
   started and clear the register when CNIC queues are stopped.

ESXi 6.5 : Version 1.0.60.0 (Nov 22,2017)
Internal FW: 7.13.3.0

Fixes:
-----------
1. Revert statemachine changes made in 1.0.58.0 related to queue state 
reporting to resolve IOVP testing issues.


ESXi 6.5 : Version 1.0.59.0 (Nov 7,2017)
Internal FW: 7.13.3.0
Fixes:
-----------
1. Revert CID leak changes (Timer scan)
2. Cont00097611:- NPAR:PSOD during reboot.
   Changes:- vmk_RegisterRebootHandler() moved it to init_module() from attach() and 
	     vmk_UnregisterRebootHandler() moved it to cleanup_module() from detach().
             So that it will get called per-module instead of per-adapter. 
3. Cont00090708: ESXi65 PSOD with "qfle3_stats_update_fuct" error


ESXi 6.5 : Version 1.0.58.0 (Oct 25,2017)
Internal FW: 7.13.3.0

Fixes:
-----------
1. Fixed a problem reported by vmware where the queues don't get restarted 
after uplink reset.
   Changes:  change the statemachine to update the SQD state variable and also 
   send vmk_UplinkQueueStop/vmk_UplinkQueueStart calls.
2. Fix CID leak issue
   Changes: Set the TM_REG_EN_LINEAR0_TIMER time scan when CNIC queues are 
   started and clear the register when CNIC queues are stopped.
3. Cont00096709:  Fix a mfw command sequence number overflow issue 
   Change: Mask off upper 16 bits of the sequence number.
   Relevance: MFW communication

ESXi 6.5 : Version 1.0.57.0 (Oct 11,2017)
Internal FW: 7.13.3.0

Fixes:
-----------
1. Cont00094192:  When the VC switch is rebooted, DCC can show "Not Linked" in
                  VCM even if the link comes up at OS level.
   Changes:  pulling CL347693.  Send DRV_MSG_CODE_LINK_STATUS_CHANGED message
             to MFW if there is a link change from analyze_link_error().  The 
             MFW will se proper link transition.
   Relevance:  Link status
2. Cont00097033: Log flooding in vmkernel logs while performing the operations 
using VI plugin
   Changes: Mask off nvm read/write logs using debug_mask.
   Relevance:  NVM read/write
   
ESXi 6.5 : Version 1.0.56.0 (Sept 08,2017)
Internal FW: 7.13.3.0
Fixes:
-----------
1. Cont00096111: Netdump is not working.
   Changes: Disable grcdump on the interface that is configured for netdump.
   Relevance: Netdump
   
2. Cont00092885: DCBX information not seen with qfle3 native driver.
   Changes: Implemented DCBX callback for get dcbx stats.
   Relevance: Management interface. 

3. worked around older firmware issue related to DRV_MSG_CODE_CONFIG_CHANGE
   message to the MFW that caused NVRAM corruption.
   
   
Enhancement:
------------
1. Added Dropless_FC (dropless flow control) support.


ESXi 6.5 : Version 1.0.55.0 (July 27,2017)
Internal FW:
Fixes:
-----------
1. Cont00095385: ESXi 6.5 management network fails to come up after blade is 
rebooted
   Changes: Making sure the queues are restarted after link up.
   Relevance: DHCP client getting ip address at boot up.
   
ESXi 6.5 : Version 1.0.54.0 (Jun 16,2017)
Internal FW: 7.13.3.0
Fixes:
-----------
1. Cont00093382: IOMMU fault seen during HeavioIOSchedulingAlgorithm test
   Changes: Avoid sending the ETH_EMPTY ramrod command to firmware. This ramrod
            is not required for Ethernet.
   Relevance: Tx traffic after MTU change and other uplink reset operations.


ESXi 6.5 : Version 1.0.53.0 (May 30,2017)
Internal FW: 7.13.3.0
Fixes:
-----------
1. Cont00093321: MC assert seen while making VFs up.
   Changes: Ensure sufficient memory is allocated to 
            account for VF statistics.
   Relevance: SRIOV

2. Cont00093700: Excessive logging of "qfle3_sp_get_next:...[vmnicX] 
                 end of spq" msgs in vmkernel
   Changes: Set proper MSG Debug Level
   Relevance: Logs

ESXi 6.5 : Version 1.0.52.0 (May 5th,2017)
Internal FW: 7.13.3.0
Fixes:
-----------
Enhancement:
------------
1. Added Bette Davis Support for FCoE and iSCSI drivers.

ESXi 6.5 : Version 1.0.51.0 (Mar 21,2017)
Internal FW: 7.13.3.0
Fixes:
-----------
1. Cont00090326: After enabling the vxlan filters in E3 qfle3 Native Driver
	    vxlan trafiic is passing through from default queue.
   Changes: Traffic will always be received on default queue as it was
	    hardcoded to use INNER_CLSS_USE_VLAN. Change it to use
	    INNER_CLSS_USE_VNI and also fix vxlan udp port endianness.
   Relevance: VxLAN filtering.

Enhancement:
------------
1. Changed rss module parameter name to 'RSS' and default queue rss to 'DRSS' as
per IOVP requirement.

ESXi 6.5 : Version 1.0.49.0 (Jan 13,2017)
Internal FW: 7.13.3.0
Enhancement:
------------
1. Fix load/unload PSOD with storage stack disabled.
   Relevance: IOVP test case.

ESXi 6.5 : Version 1.0.48.0 (Jan 11,2017)
Internal FW: 7.13.3.0
Enhancement:
------------
Modifying the cnic_if.h interface files to match qcnic.


ESXi 6.5 : Version 1.0.47.0 (Dec 22,2016)
Internal FW: 7.13.3.0
Enhancement:
------------
Add check for qfle3_tq_stopped flag during TX-Completion
interrupt processing.

ESXi 6.5 : Version 1.0.46.0 (Dec 7,2016)
Internal FW: 7.13.3.0
Fixes:
-----------
1. Cont00089681: Enhancement to display MAC during queue creation/deletion.
   Change: Enhanced Debug messages for "QFLE3_DBG_QUEUE-0x00001000" debug
           settings.
   Relevance: Logs

2. Cont00088602: Information in AHS log is incorrect.
   Change: Enable HW stats update for PF. It was only enabled for SRIOV.
           Populate fcoe and iscsi stats along with ether for DRV_STATUS_DRV_INFO_REQ.

3. Cont00089904: Network Connectivity lost when esxcli network nic up -n
		vmnic4* command was run.
   Change: In MF_SD mode, added fix to not change link state of other NPAR
	   functions when link operations are performed on one of the 
	   asscociated NPAR function.
   Relevance: Switch dependant NPAR mode. 

4. Cont00089186: VMkernel traffic thread breaks when other multiple 
                 threads are started.
   Change: Remove check for tq_stopped state when completion interrupt 
           is received in case when Queue is stopped as a result of getting 
           too full.
   Relevance: Transmit Queue.
           

ESXi 6.5 : Version 1.0.45.0 (Nov 11, 2016)
Internal FW: 7.13.3.0
Fixes:
------
1. Cont00089646: Continuous messages in Vmkernel logs for the nic under test in E3 native 
driver bundle
	Change: change the message to a debug msg that can be turned off/on with 
	module parameters.

2. Cont00089693: NIC driver fix for this CQ.  FCoE Driver enumeration 
problem.
	Change: Make sure ISCSI or FCOE is disabled if there is no valid MAC 
	address.

3. Cont00089894: E3-Native-FCoE: fcoe adapter shows WWN of PF0 in NPAR mode
	Change: Read the WWN Port name and node name from proper location for MF 
	mode.
	


ESXi 6.5 : Version 1.0.44.0 (Oct 19, 2016)
Internal FW: 7.13.3.0
Fixes:
------
1. E3 Multi Function Mode with FCOE init ramrod not completing.

	Change: Initialize Status Block even if only FCOE is supported.

	Relevance: FCOE.
	

ESXi 6.5 : Version 1.0.43.0 (Sept 30, 2016)
Internal FW: 7.13.3.0
Fixes:
------
1. Cont00087673: BFS with openfcoe driver and qfle3 inbox driver
	         fails on second port

   Change: Integrate firmware 7.13.3.0. Removed initialization of
		 PXP2_REG_PGL_INT_TSDM_* registers which are chip
		 common. Filled PXP_REG_HST_INBOUND_INT with zeros
		 before protocol-specific initialization.

   Relevance: Boot from SAN.

2. Cont00089061: [E3-NIC+iSCSI]: While nic driver with "intr_mode=1"
			(IRQ), PSOD is observed

   Change: Disable creation of CNIC logical device when intr_mode is not MSI-X

   Relevance: Interrupt Mode

3. Cont00089038: [E3-NIC+iSCSI]: Logs flood with "qfle3_xmit_pkt:656:
			[vmnic13] To defrag TSO pkt2" msgs

   Change: added a private counter for TSO segments out of hardware limit and 
   change message to debug message.

   Relevance: TSO

4. Cont00088989: E3 Native FCOE: Getting zero throughput & I/O error during
			FCOE traffic.

   Change: Kick start DCBX negotiation in dirver for FCOE.

   Relevance: FCOE


ESXi 6.5 : Version 1.0.41.0 (Sept 15, 2016)
Internal FW: 7.13.1.0
Fixes:
------
1. Cont00087653: VM's dmesg showing SET MAC failed while doing 
                 MTU change on VM

   Change: Corrected SRIOV BC processing for MTU change on PF.

   Relevance: SRIOV

2. Cont00088907: PSOD while loading driver after unload

   Change: Avoid using unitialized queue objects.

   Relevance: SRIOV

3. Cont00088963: Uninstall::Driver::VIB failed for qfle3.

   Change: Change VIB properties 'live-remove-allowed' to 'False'.

   Relevance: VIB Remove/Installation.

4. Cont00088689: E3 Native NIC: PSOD encountered when port down and then reset

   Change: Avoid starting Netpoll on deactivated queues.
   
5. Cont00087536: ESXi6.5-unnecessary vmkernel logs when we run vxlan traffic.

   Change: Reduce the logging level.

   Relevance: Encapsulation.

Enchancements:
-------------
1. Merge helper thread related fixes from inbox driver.

ESXi 6.5 : Version 1.0.40.0 (Aug 28, 2016)
Internal FW: 7.13.1.0
Fixes:
------
1. Cont00087426: QCC Showing "Link down" in adapter info even when link is
                 connected using ESXi 6.5.0

   Change : Provide link up/down inforamtion to mgmt instead of link status.

   Relevance: Mgmt.

2. Cont00088480: PSOD IOMMU Fault detected after enable SRIOV over NPAR.

   Change : Avoid sending statistics request to firmware before statitics
	    structures are initialized.

   Relevance: Statistics.

3. Cont00087388: PSOD hits when MTU is modified.

   Change: Don't send statistics requests when MTU is being changed.

   Relevance: Statistics and MTU change.

4. Cont00088391: PSOD when running jumbo traffic with MTU size 9000.

   Change: Fixed uplink reconfiguration.

   Relevance: Jumbo frames.

5. Cont00087639: E3 Native iSCSI: PSOD when enabling jumbo frame

   Change: same as 4 above.

   Relevance: iSCSI jumbo frame.

6. Cont00088313: PSOD while bringing down the test NIC and doing the firmware reset in 
qfle3.
   Change: Fixed uplink reconfiguration.

   Relavance: NIC reset.

7. Cont00088326 Geneve: ESXi6.5 Beta3: PSOD is seen while creating vSwitch using latest 
encap

	Change: Fixed uplink reconfiguration.

	Relevance: Geneve.

8. Cont00087587: psod after driver installation and reboot.

   Change: Fixed driver unload bug.

   Relevance: Reboot.


Enchancements:
-------------
1. Added VF Device ID for 57840 adapter.
2. Merge fixes from inbox driver.

ESXi 6.5 : Version 1.0.27.0 (Aug 17, 2016)
Internal FW: 7.13.1.0
Fixes:
------
1. CQ#88395 - Huge log flooding in vmkernel logs. VMkernel 
              logs are printing every TX and RX packet.
   
   Change : Commented Logs in the data Path

   Relevance: Logs

2. CQ#87571 - Correction in VF Description

   Change: Correct VF Description in qfle3_devices.py

   Relevance: SRIOV

3. Cont00086027: IPV6 ping from VF to VF not working.
   
   Change: Calculate multicast bin correctly.

   Relevance: SRIOV Multicast traffic.

4. Cont00087667: Ping between VF to PF of same physical function doesn't work.

   Change: Setup filter on Tx side and fill desct MAC in BD.

   Relevance: SRIOV Tx Switching.

5. Cont00087058: Server PSOD in qfle3_tq_free in SRIOV Configuration.

   Change: Add check for sm inside qfle3_quiesce_dev.

   Relevance: SRIOV

Enchancements:
-------------
1. Enabled firmware dump collection after system crash by default.


ESXi 6.5 : Version 1.0.26.0 (Aug 3, 2016)
Internal FW: 7.13.1.0
Fixes:
------
1. This is a NIC only driver with removed storage 
   support built with ESX6.5 Beta3 DDK.

   Change : Commented QFLE3_CNIC switch from scons file

   Relevance: Storage

2. Fixed SRIOV crash caused due to Stats collection
   due to storage changes.

   Change : Corrected no. of eth queues for stats collection.

   Relevance: SRIOV

ESXi 6.5 : Version 1.0.25.0 (Aug 1, 2016)
Internal FW: 7.13.1.0
Fixes:
------
1. Fix CNIC resource release issue on unload. 

ESXi 6.5 : Version 1.0.24.0 (July 28, 2016)
Internal FW: 7.13.1.0

   Updated Driver Version for ESX6.5 Beta 3 DDK. 

ESXi 6.1 : Version 1.0.23.0 (July 27, 2016)
Internal FW: 7.13.1.0
Fixes:
------
1. Cont00086831: Need of Vxlan stats implementation in qfle3 driver.

   Change : Added driver stats for TSO, CKO and encapsulated pkts.

   Relevance: Driver stats.

2. Cont00087986: Geneve - PSOD is seen while running Traffic (Tx).

   Change : Remove ASSERT(ip4h->checksum) as zero is a valid checksum.

   Relevance: Encapsulated pkts.

3. Cont00087683: E3:Enabled SRIOV on one Adapter, gets enabled on 
                  both the adapter in host.

   Change : Use Relative PF ID to Index on max_vfs

   Relevance: SRIOV

ESXi 6.1 : Version 1.0.22.0 (July 11, 2016)
Internal FW: 7.13.1.0
Enhancements:
-------------
1. Add support for CNIC driver. (FCOE & iSCSI)

ESXi 6.1 : Version 1.0.21.0 (July 08, 2016)
Internal FW: 7.13.1.0
Enhancements:
-------------
1. Add support for Internal loopback (MAC and PHY).
2. Add support to display PHY version (if exists) along with FW version.
3. Add support for physical port identification test using LED.
4. Support for Geneve offloads - TxCKO/LSO/RSS/Geneve port update
5. Add support for PHY upgrade using mgmt tool.

ESXi 6.1 : Version 1.0.20.0 (June 20, 2016)
Internal FW: 7.13.1.0
Fixes:
------
1. Cont00086832: tx_queue number is displayed as an absolute number.

   Change : Display relative tx queue number.

   Relevance: Tx queue stats.

Enhancements:
-------------
1. Added support for default queue RSS.
2. Added support for VXLAN Rx filters.
3. Added support for on-chip MFW dump detection.
4. Added support for VF statistics.


ESXi 6.1 : Version 1.0.19.0 (June 04, 2016)
Internal FW: 7.13.1.0
Fixes:
------
1. Cont00085061: Ring size not changing. 

   Change: Fixed a minor bug in calculation of number of pages required for a
	   particular ring size.

   Relevance: Rx/Tx ring size change using "esxcli network nic " command.

2. Cont00086812: PSOD with 1.0.18 driver in certain MF configurations. 

   Change: Fixed a minor bug in identifying the protocol type for PFs.

   Relevance: UFP.


Enhancements:
-------------
1. NPAR: Added support for dynamic bandwidth configuration with esxcli command.
2. Merge following changes from vsphere 2016 inbox driver:
	a) Fix error path bugs found during DDV test ei_loading.
	b) There is a problem in qfle3 driver LRO code, if tpa_info->len_on_bd
	   is the full length of a received frame, it may not a large TCP
	   packet, we should not set the MSS for such a packet.
	c. Although driver has already announced the maximal filter number
	   per-netq, netstack will still apply new filter on a full queue.
	   Driver should return failure even for the default queue. Thus
	   vmkernel know the right action of driver.
	d. Add error handling for vxlan packet. Drop or mark them not a vxlan
	   packet when there is error.
	e. Properly handle vxlan packet with inner L4 type VMK_PKT_HEADER_L4_ICMPV6.
	f. Checking parameter range in coalesce callback, and fix indent in tx
	   callback.
	g. Delete some debug code in tx path.
 
ESXi 6.1 : Version 1.0.18.0 (May 24, 2016)
Internal FW: 7.13.1.0
Fixes:
------
1. Cont00086105: After sending Pause frame ping doesn't stop.

   Change: Integrate storm firmware 7.13.1.0 which has the ability to detect
        malicious VFs sending pause frames.

   Relevance: SRIOV and pause frames.

2. Cont00086429: (E3 qfle3 SRIOV over NPAR) after enabling NPAR+SRIOV setting 
	max_VF parameter as 16 system crashed PSOD

   Change: Handle SRIOV Enablement failure due to server resource limitation.

   Relevance: SRIOV + NPAR

3. Cont00085969: Vmkerenl logs flooding with msgs "qfle3_attn_int_deasserted0"

   Change: Call elink fucntion to handle SFP module detection attention.

   Relevance: Link state.

4. Cont00086704: Driver doesn't reload after setting the intr_mode to 1

   Change:  Skip vmk_NetPollInterruptSet for MSI and legacy interrupts.

   Relevance: Single interrupt modes.

5. Cont00086024: (E3 qfle3 SRIOV) Windows VM crashed while trying to install evbd

   Change: Handle Windows VF to PF sequence of Queue Stop and FLR correctly.

   Relevance: SRIOV


Enhancements:
-------------
1. SRIOV: Fixed CRC messages observed on VF w.r.t. Back
	Channel TLV Exchange. 

2. Netqueues: Make 8 Rx queues and 8 Tx queues default setting.


ESXi 6.1 : Version 1.0.17.0 (April 27, 2016)
Fixes:
------
1. Cont00086017: Adapter which support Vendor specific switch dependent partitioning are not claimed by qfle3 driver.

Enhancements:
-------------
1. SRIOV: Added support for PVLAN, GVLAN and Vf MTU Change check.


ESXi 6.1 : Version 1.0.16.0 (April 15, 2016)

1. First driver release for qfle3
