Thursday, August 14, 2014

H323 traffic failing to traverse a Fortigate firewall

Had a scenario recently where a Polycom video conferencing device just wouldn’t work when sat behind a Fortigate firewall.  This was despite all the necessary TCP ports being forwarded to the device, as verified by Polycom support.

What we were seeing is that one could dial the VC but it would just ring and never make the connection.  Time to debug the traffic on the Fortigate – this is what I saw:

id=13 trace_id=74 msg="vd-root received a packet(proto=6, x.x.x.x:1720->x.x.x.x:63665) from lan."
id=13 trace_id=74 msg="Find an existing session, id-02237475, reply direction"
id=13 trace_id=74 msg="SNAT x.x.x.x->x.x.x.x:1720"
id=13 trace_id=74 msg="run helper-h323(dir=reply)"

The “run helper” sequence kicked in as soon as one attempted to pick up the call on the VC.  In Polycom’s case they suggest explicitly disabling any h323 helpers, so that is exactly what I did.  I did it like so:

  1. From the cli, execute “config system session-helper”.  This will give you the following output (below is redacted)
    edit 2
            set name h323
            set port 1720
            set protocol 6
        edit 13
            set name sip
            set port 5060
            set protocol 17
  2. Now delete these helpers by executing
    config system session-helper
    delete 2
    delete 13
    end
  3. Enter the following commands:
    config system settings
    set sip-helper disable
    set sip-nat-trace disable
  4. Lastly we disable RTP processing:
    config voip profile
    edit default
    config sip
    set rtp disable

Your h323 and SIP traffic should now traverse your Fortigate without issue.  In my experience this has only happened with Polycom devices, Microsoft Lync works fine without modification.