Configuring a 4G USB modem on a Fortigate Firewall

Fortigate is a deserved market leader in the UTM space, but as with all products there is always room for improvement.  Point in case is getting a 3G / LTE connection via a USB dongle up and running.  There is minimal support in the GUI, and they’ve removed a lot of options in FortiOS 5.

Fortunately for an old Cisco hack like myself we’ll always have the CLI to fall back to so below are the steps to get it up and running.  Note that the below is for a device that is on the Fortigate HCL.  If yours is not then you’ll have to configure that first.

config system modem
    set status enable (Enables the modem functionality)
    set pin-init "at+cpin=1111" (If you have PIN security, I tend to disable the SIM PIN check)
    set dial-on-demand enable (self-explanatory)
    set phone1 "*99#" (replace *99# with whatever your dial-up number is)
    set extra-init1 "at+cgdcont=1,\"ip\",\"internet\"" (replace internet with your APN)
    set altmode disable (enable if you're in China, rest of the world disable)
end

VLAN Up, Line Protocol Down

Creating a VLAN and a SVI

Today, not for the first time, I was asked to troubleshoot a issue with an SVI (Switch Virtual Interface) not coming up.  The customer had created a SVI like so:

switch# conf t
switch (config)# int vlan 10
switch (config-if)# ip address 192.168.10.1
switch (config-if)# no shut

The problem was that the SVI stayed down, and the VLAN did not even show up when you do a “sh vlan”.  Why did this happen?  Because the customer created the SVI without first creating the Layer 2 VLAN for it to live in.  Here is what the customer should have done:

switch# conf t
switch (config)# vlan 10 ***this creates the layer 2 VLAN***
switch (config-vlan)# name VLAN_NAME
switch (config-vlan)# exit
switch (config)# int vlan 10 ***this creates the layer 3 SVI for the VLAN***
switch (config-if)# ip address 192.168.10.1 ***assigns the IP and netmask to the SVI***
switch (config-if)# no shut

Now if we do a “show vlan” it should show as active.  Whilst we’re on this topic, let’s have a look at what exactly is required for a SVI to come up and be ping-able on your network.

Requirements to get a SVI to the Up/Up state

An SVI must meet all of the below requirements to transition to the “Up/Up” state

• Its VLAN must exist and show as active in the VLAN database “sh vlan”
• One switched port in the VLAN (can be access or trunk) must be up
• The port must be in a STP forwarding state

Handy WMI GPO filters

I thought I’d document a couple of WMI filters I’ve found handy when deploying software via Group Policy Objects (GPOs).  Windows Management Instrumentation (WMI) filters allow you to dynamically determine the scope of GPO based on attributes of the target computer.

When a GPO that is linked to a WMI filter is applied on the target computer, the filter is evaluated on the target computer. If the WMI filter evaluates to false, the GPO is not applied (except if the client computer is running Windows 2000, in which case the filter is ignored and the GPO is always applied). If the WMI filter evaluates to true, the GPO is applied.

Target Windows 7 32-bit machines

select * from Win32_OperatingSystem WHERE Version like "6.1%" AND ProductType="1" AND NOT OSArchitecture = "64-bit"

Target Windows 7 64-bit machines:

select * from Win32_OperatingSystem WHERE Version like "6.1%" AND ProductType="1" AND OSArchitecture = "64-bit"

Target any 32-bit OS:

SELECT AddressWidth FROM Win32_Processor WHERE AddressWidth ='32'

Target any 64-bit OS:

SELECT AddressWidth FROM Win32_Processor WHERE AddressWidth ='64'

Target Windows 7, Vista, Server 2008, Server 2008 R2 

select * from Win32_OperatingSystem where Version like "6.%" '

Target Windows 7, Server 2008 R2

select * from Win32_OperatingSystem where Version like "6.1%" '

Target Windows Server 2003

select * from Win32_OperatingSystem where Version like "5.2%" '

Target Server 2003, but not domain controllers

select * from Win32_OperatingSystem where Version like "5.2%" and ProductType="3" '

Target Server 2008 and 2008 R2, but not domain controllers

select * from Win32_OperatingSystem where Version like "6.%" and ProductType="3" '

Note:  Substitute ProductType="3" ' with ProductType="2" ' to just target domain controllers.

Schedule a Reload on a Cisco Router

Who hasn’t been in the situation where you have to make changes to a Cisco devices that you do not currently have physical access to. Fun and games until you fat finger something and lose access to the device. 

An easy way around this is to schedule a reload of the device before you start making changes.  That way if something does go wrong the device will reload and come up as before.  Of course if the reload is not required you can cancel it.  Below is the steps needed to achieve all that.

Cisco_Switch#copy run start
Destination filename [startup-config]?
Building configuration...
[OK]
Cisco_Switch#reload in 015
Reload scheduled in 15 minutes
Proceed with reload? [confirm]
Cisco_Switch#reload cancel
Cisco_Switch#

***
*** --- SHUTDOWN ABORTED ---
***

Cisco_Switch#

Microsoft Network Load Balancing, and how to make it work with a Cisco Nexus

Microsoft Network Load Balancing has one or two “interesting” issues due to the way it does networking.  More specifically, it utilises a Multicast MAC address with a Unicast IP address.  Someone much smarter than me explains the ins and outs of the issue here. 

This issue manifests itself in that clients outside of the cluster’s LAN cannot connect to the IP.

Now that we understand the issue,  how do we resolve it?  The fixes listed in the article are all valid, however we can get this working by tweaking the config on our Nexus.

interface Vlan10
  no ip redirects
  ip address 192.168.10.1/24
  ip arp 192.168.10.100 03BF.0A03.00CF
  description Server Farm
  no shutdown

Your clients should now be able connect to the cluster IP address without issue.

Updating the firmware (DDOS) on a DataDomain

EMC has done a great job with DDOS 5.2, as such I’ve had to do a couple of upgrades recently.  The upgrade manual is quite a tome, so this is my notes, distilled.  I’ve added a couple of extra steps just to make sure there are no error conditions that can hamper the upgrade or the operation of the DataDomain.

Pre-Upgrade checks

Review the output of the below commands for any errors and warnings.

1. Reboot the DataDomain (I do it just because)
2. ssh into the DataDomain and issue the commands listed below
3. enclosure show powersupply 1
4. alert show current
5. reg show config.net (confirm the following settings)
1. noauto.enabled
2. noauto.speed
3. noauto.full_duplex
6. ifgroup
7. disk show state
8. disk show reliability-data
9. enclosure show all
10. enclosure show topology
11. system show hardware
12. system show ports
13. filesys status
14. log view debug/platform/kern.info

Upgrade Steps

Well after all that the steps to actually upgrade the DDOS is a bit of a let-down.

1. Copy your DDOS .rpm to /ddr/var/releases
1. You can do this via a NFS mount or you can upload via the GUI
2. ssh into the DataDomain and issue the commands listed below
3. system upgrade 5.2.2.4-370754.rpm
4. Monitor progress with “system upgrade status” or “log watch debug/platform/infra.log”

Post-Upgrade Steps

I guess strictly speaking these steps are not necessary, as with the pre-upgrade steps.  I do it just to ensure I have a nice neat base to work from

1. ssh into the DataDomain and issue the commands listed below
2. ost disable
3. ost enable
4. nfs disable
5. nfs enable

And that is all there is to it, mostly.

Cisco Port Configuration Best Practices

I run into inconsistent network configurations wherever I go, from customers who just lets everything live in the native VLAN to ones who horrendously over-complicate things.  For the majority of Cisco deployments I have a simple set of configuration standards I adhere to.  You will always get corner-case requiring special configurations, but I find that the below works very well for most use cases.

Standard Access Port Configuration

This configuration is applied to ports connecting to standard end-user equipment, like PC’s, printers etc.
switchport mode access
switchport access vlan 5
spanning-tree portfast
spanning-tree bpduguard enable

The first two lines are self-explanatory, it’s an access port living in vlan 5.  Portfast is something we configure on an access port, which tells STP not to bother and just put the port in a forwarding state, as opposed to taking time to go through the listening and learning states. 
BPDU’s are basically STP messages exchanged between switches, therefore a BPDU is not something we expect to see on a normal access-port.  BPDUguard tells the switch that if it receives a BPDU (for example because someone connected an unauthorized switch), it should shut the port down.  Finally BPDUFilter tells the switch to not send or receive BPDU’s on ports configured as portfast.

ESXi Host Port Configuration

ESXi uses internal vSwitches and PortGroups which allows for VM’s running on one host to reside in different VLANs
switchport mode trunk
switchport trunk allowed vlan 5-10,200
spanning-tree port type edge trunk
spanning-tree bpduguard enable
Fairly straightforward, we configure our link as a trunk which carries vlan 5 to 10 and 200.  We then tell the switch that even though it’s a trunk we’re no connecting to another switch so no need to worry about STP on the port.  Lastly bpduguard protects us against incorrect cabling – if the port receives a BPDU (which will happen if you inadvertently hook it up to another switch) it will shut down.
 

Standard Trunk Port Configuration

In the Cisco world a trunked link is a link that carries multiple VLANs.  Not to be confused with link aggregation, which in Cisco parlance is called an Port-Channel.
switchport mode trunk
switchport trunk encapsulation dot1q
switchport trunk native vlan 10
switchport trunk allowed vlan 10-15
Both sides of the trunk needs to have the same default VLAN, by default the native VLAN is VLAN 1, but in all but the simplest deployments you will have to change this.  We can also do VLAN pruning on our trunks, that is only carry certain VLANs accross our trunk.

Port-Channel Configuration

It is possible to aggregate multiple links between two switches and treat them as a single link.  This gives us link redundancy and bandwidth increases.  As a rule we do not use LACP to ensure compatibility with, for example, vSphere vSwitches.
interface Port-channel10
switchport trunk encapsulation dot1q
switchport mode trunk
Once again simplicity is the name of the game.  We create the interface, set the encapsulation (not necessary, strictly speaking since Cisco defaults to dot1q) and set it as a trunk.  Of course it does not need to be a trunk link, this is optional.

Issues and Workarounds – vSphere Site Recovery Manager with the NetApp Storage Replication Adapter 2.01

I have just completed a project where I had to Install and configure VMware vSphere Site Recovery Manager.  Storage was provided by NetApp FAS and V-Series filers, thus I had to use the NetApp provided Storage Replication Adapter.  As of the time of this writing the latest version was 2.01.  True to form I ran into a couple of bugs, which took a bit of figuring out.

Unable to add a controller: “Error: SRA command ‘discoverArrays’ failed”

Execute the following commands on your filers

• options httpd.admin.enable on
• options httpd.enable on
• options httpd.admin.ssl.enable off

Error when adding an Array Pair: “Internal error: std::exception 'class Dr::Xml::XmlValidateException' "Element 'SourceDevices' is not valid for content model: '(SourceDevice,)”

There are two solutions to this issue

• Downgrade back to NetApp SRA version 2.0.0
• Manually include the lists of volumes you want discovered by the SRA.  You’ll need to do this on both controllers in the pair.

This is a documented bug

Reprotect Job fails after recovering to Disaster Recovery Site

The SRM / SRA timeouts seems a bit aggressive to me.  This is highlighted when you do a reprotect on a failed over Protection Group.  Part of the task sequences is to reverse the direction of replication, but this fails consistently due to the SRM not waiting long enough for this reversal to take place.

You can kludge it by:

• Re-running the reprotect until it works
• Manually refresh the Array Manager while the Re-Protect job is running

Recovered DataStores have snap-xxx prefixes

More of a cosmetic irritant than a true bug, I wanted this fixed nonetheless.

1. Within SRM, right-click your site and select Advanced Settings
2. Click StorageProvider
3. Select the storageProvider.fixRecoveredDatastoresNames check box

Tip

I would suggest increasing your SRM SAN provider timeout settings to something a bit more sane, like double.  Instructions can be found here.

Also make sure that the ALUA settings on your iGroups in both the protected and recovery sites are the same.