1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

QOS

Discussion in 'Tomato Firmware' started by Jason Woringen, Jan 16, 2017.

Tags:
  1. Jason Woringen

    Jason Woringen Network Newbie Member

    I thought I would post some useful information about a VERY VERY VERY simple quality of service/active que management solution. Most people tend to overcomplicate their configurations and really don't understand the methods behind what qos actually invokes. My particular setup consists of Charter Cable in Louisiana. Bufferbloat tests via dslreports.com, betterspeedtest.sh, and other mechanisms all confirmed extremely high levels of bufferbloat ranging from 250ms upwards to 700ms before this. That's ridiculous needless to say. Many people know about bufferbloat, even more don't. My household consists of myself and 1 roommate with the occasionally weekend guests of 6+ all steaming, gaming, and surfing to which the my router happily accepts the pain and laughs. A+ on dslreports.com for quality, bufferbloat, and bandwidth after. I apply these same settings to My R7000's on 6 tourist bars in the New Orleans French Quarter Area Streaming 6 Webcams, a Jukebox, upwards of 200 guests doing everything from facebook to downloads using a 50 down 10 up connection out there. What is my setup you say?

    Nighthawk R7000
    Tomato Firmware 1.28.0000 -138 K26ARM USB AIO-64K Overclocked @ 1200,800
    Cisco DPC3216 DOCSIS 3.0 2-PORT Voice Modem DPC3216

    My Qos settings are as follows:

    Enable QoS - enabled
    Prioritize small packets with these control flags ACK SYN FIN RST <-all enabled
    Prioritize ICMP - checked
    No Ingress QOS for UDP - enabled
    Reset class when changing settings - enabled
    Default class - Normal
    Qdisc Scheduler - fq_codel


    My average speeds measured are 65Mbps+ && 5Mbps+ 13-1 ratio
    Now less 20% from both figures [52Mbps && 4000Mbps] 13-1 ratio

    2 classes named Fast and Normal. 50% for each Minimum and Maximum. This means 50% reserved for the minimum and 50% for the maximum even if another class is using it. Acks for web browsing, gaming, even voip use small amounts of bandwidth with small packets. Some will argue that voip uses udp for voice but tcp is play use as well for other mechanisms. Another suggestion would be to use 30% for Fast and 70% for Normal. Whichever suits you.

    Under Classification>OutBound Direction (delete all rules first)

    Match Rule Class Description #

    TCP/UDP
    Transferred: 0 - 512KB class: Fast Description: 512KB OR LESS 1

    TCP/UDP
    Transferred: 512KB+ class: Normal Description: 512KB OR MORE 2


    **It is this simple. No ip addresses, ports, or filters applied except what is above. A reboot maybe required. Test it out. Let me know what you think. Some routers may see better results using 256kb splits. I should mention my conntrack/netfilter settings are changed but that's a whole other discussion! Try this first and then compare the results.






     
    Last edited: Jan 21, 2017
    visceralpsyche likes this.
  2. cloneman

    cloneman Networkin' Nut Member

    This does in fact work, and I've vouched for less is more approach to number of QoS rules for quite some time.

    I won't say it's the best setup but its far better than doing nothing, and it's easy to understand. You can also increase the maximums of the Normal class, as the fast class almost always able to borrow from normal even if normal is set to 100%. (though I'd advise against 100% if torrent or steam downloads are involves)


    I started with a similar approach, and then I added little things like VoIP and icmp to #1 above both fast and normal. 10% min and max.

    Conntrack magically classifies RTP VoIP traffic for you if you put DEST 5060 on high. (sometimes, for some providers)

    Anyway my screenshot is out there in the main thread. I Basically leave most traffic in default, push VoIP/icmp on top, and gradually kick out large transfers from the normal class (I have 3 levels below normal, big transfer, huge transfer, and torrent)

    I also force Chromecast to remain in normal/default, even though it's a large transfer, it's an important one.

    So yeah, in your setup, I'd lower the minimums as you only have a budget of 100% and you're using it all in 2 classes, doesn't give you room to add more stuff later. The borrowing mechanism allows you to set higher maximums and lower minimums than you currently have. The only reason I see to justify having a 50% maximum is to deal with steam downloads that make a mess if you let them use more than 50%

    Also, most torrent clients in advanced settings allow you to change the main port and also the source port range for connections. Generally you can configure the torrent client for each person in the home so they stick to their ports. They won't all respect it, they'll switch app on you or format their PC and forget it about it, but it's worth a shot - some people are greatful to be compliant and enable a better experience for everyone by being able to sort torrent traffic more easily to "low". People who repeatedly don't comply can have all their traffic set to low :)

    The future of QOS is similar to these ideas. There's some untapped potential in packet size as well. I'd like to see sorting by packet size in the future. Not sure, but edgerouter's smartqueue might have implemented this.
     
    Last edited: Jan 16, 2017
  3. cloneman

    cloneman Networkin' Nut Member

    One more thing aside. Your 50/50 approach with only 2 classes goes along the lines of a strategy that can help people with older third-party QoS systems.

    Example: You're on a firmware or router that doesn't have borrows on the downstream side of things (No Global limit on the download side, and no minimums on the download side, etc). Such as, AsusWRT / Merlin, Old Tomato versions, and many others.

    Solution: You can setup a 2-class system, (like the OP) where a lot of bandwidth is taken off the table, such as the 50/50 approach. This creates a virtual 'global maximum' for downloads that cannot not exceed 100% when multiple classes are in use. You might have to go 40/50 to stay below 90%, etc.
     
  4. Jason Woringen

    Jason Woringen Network Newbie Member

    Excellent explanation cloneman. This is like I stated earlier a simple approach to get folks to understand how things are working at the basic level. It's a not the perfect solution, but a viable solution for the average household that doesn't need 20 rules to classify traffic. In my experience the average users are web-surfing, messaging, streaming music/videos, and downloading files with the exception of bittorrenting. Our medium to advanced crowd can obviously do more to their setup. I implemented this very configuration in a friends house who was complaining about videos buffering while her son plays Call of Duty on their Linksys EA6700. I've received a home cooked lasagna and many praises with 3 more clients. I do think a beginner and advanced gui function with this sort of selection would be a nice addition. It could possibly alleviate a lot pain with questions about qos.
     
    Toastman likes this.
  5. Jason Woringen

    Jason Woringen Network Newbie Member

    My current settings I'm testing are the same except Upload is 75% Fast and 25% Normal. Download is 25% Fast and 75% Normal. Seems to be working excellent.
     
  6. Monk E. Boy

    Monk E. Boy Network Guru Member

    For incoming I would normally set the maximum to 100% since you don't actually control ingress traffic. When any packet is rejected (or dropped) by the firewall it will get resent by the receiver.

    Packets originating from the LAN side of the firewall don't matter much, the packet will get resent using the gobs of available bandwidth within the interior network (assuming a minimum of 1Gb). But the WAN is connected via a relatively low bandwidth link. When a packet is resent over that link, it uses up bandwidth that could otherwise go towards normal traffic. Think of a rejected packet as a packet that travels over the WAN link two or more times. Reject enough packets and this can create a DDoS like condition, since a rejection typically involves a packet being sent out alerting the sender that the packet was rejected, then it resends a packet, it gets rejected... on and on. A timeout is a little different since there's no outbound packet but the incoming packet still gets re-sent.

    In other words, 50/50 outgoing is A-OK, but unless you control the other end of the WAN connection, you will want to set incoming classes to 100% max to avoid rejecting (and resending) packets.

    I'd suggest trying this out and seeing if there's any change in your situation. I would guess it won't, at least if your internet connection is like most U.S. consumer connections with a lower upstream connection than its downstream connection (e.g. 70Mb/5Mb). In those situations if you can maintain control over your upstream connection you should be in good shape.
     
  7. cloneman

    cloneman Networkin' Nut Member

    With tomato ingress QoS, no message is sent out to indicate a packet is rejected afaik. It acts as a policer which drops traffic to pretend it has run out of bandwidth, encouraging the other side to slow down. If your HTTP traffic is shaped to 25mbit, the policer will drop all packets that exceed this amount, and your other applications will not have their traffic dropped.

    The reason most people leave incoming "max columns" at 100% is because Tomato QoS uses an IMQ/IFB borrowing system, so it will automatically begin policing/dropping when a higher priority class of traffic needs it. If VoIP needs 100kb/s, it will automatically drop 100kb/s from the lowest priority class (assuming a congested link - nothing gets dropped if the link has unused capacity)
     
  8. visceralpsyche

    visceralpsyche Networkin' Nut Member

    Thanks for the setup advice Jason :)

    I'm a little confused - am I supposed to have the 50% 100% as shown in the screenshot below, for both outbound and inbound, or should I have it all 50% 50% 50% 50%?

    And in your example above, should I set Fast to 75% 100% and Normal 25% 100% for outbound/inbound, or Fast 75% 50% Normal 25% 50% outbound/inbound?

    Thanks for any clarity!

    Cheers,

    Paul :)

    [​IMG]
     
  9. cloneman

    cloneman Networkin' Nut Member

    It's subject to personal preference. 50min , 100 max , will work in a 2-class system. If torrent traffic falls into your 'normal' bucket, I would decrease the max from 100% to perhaps 80%. The goal here is to keep traffic that is "disrespectful" to Downstream policers in better check.

    Or introduce a BelowNormal class for problematic traffic. a 3-class system would look something like this

    Fast 40 100
    Normal 40 100
    BelowNormal 10 80

    In Classification page for BelowNormal you would have your torrent client's main port, as well as the Outgoing port range of about 100 ports that you manually pre-configure in your torrent client's advanced settings.

    Note that Steam game downloads implements a ridiculous system that may cause packet loss unless heavily throttled below 50%. I have no solution for this yet. (Either limit it in Steam's GUI or accept that you'll have packet loss when you download a game from them). Or You may decide that a hard 50% MAX BelowNormal limit is an acceptable compromise to allow many users to use torrents or steam at half line speed simultanously with no impact on everyone else.
     
    visceralpsyche likes this.
  10. visceralpsyche

    visceralpsyche Networkin' Nut Member

    Do the inbound and outbound speeds/percentages need to be the same?

    Am testing your suggestions now @cloneman :)

    Thanks!
     
  11. Jason Woringen

    Jason Woringen Network Newbie Member

    Screen Shot 2017-01-27 at 10.45.03 AM.png Screen Shot 2017-01-27 at 10.45.36 AM.png Screen Shot 2017-01-27 at 10.46.05 AM.png [​IMG][/url][/IMG]
    The inbound and outbound percentages are the same in my example for simplicity. the minimum percentages must equal 100% and not more or less for that matter. If you go over, the system will overwork itself trying to constantly manage the pie if the line gets saturated. At 50% the system becomes almost instantly to responses to packet utilization. [​IMG]If you go under unexpected shifts of bandwidth will go unmanaged to another rogue application. I'm also using 80% of my tested download and upload speeds to manage the bufferbloat. Remember if one class needs more it will borrow more. This is essentially the same effect as making at least 5 classes with small to large priority such as web browsing, dns, time, email, youtube, downloading, etc except without all the leg work. It prioritizes small packet/low-bandwidth applications first, then the large packet/high-bandwidth applications second. I've tested this theory while downloading windows updates, a gamer playing cod, another kid playing on facebook, another watching netflix, and another streaming spotify. Every answer response to the question of "Do you notice any difference?" was "Works Great". I've also implemented this in workspaces for heavy traffic at 6 public bars with the same results. My OpenVPN is fast and responsive and no one complains. I've tried different percentages but this seems to be the best for the amount of bandwidth. If I had a fiber line with much higher upload speeds I could vary the percentages, but unfortunately this isn't that case.
     
    Last edited: Jan 27, 2017
    visceralpsyche likes this.
  12. Jason Woringen

    Jason Woringen Network Newbie Member

    You'll notice that while the line IS NOT saturated that one class may consume over 90% of the bandwidth in the pie graph. This is normal behavior. Large packets from different sources means the top priority still gets first priority. Large packets produce a temporary calculation on bandwidth allocation but still allows the small packets to go first using a smaller amount of bandwidth. This happens very quickly and is most likely never noticed by any user. Although P2P/bitorrents generally use small packets they create more connections slowing down resources in the router/data line. Small transfers add up to large transfers resulting in a greedy pig that never gets full. Try opening over 100 webpages full of flash content, java, and constantly refreshing and that's the idea of what's happening. The minimum column is for reserving a certain amount of bandwidth at all times. The max column's function is to limit to a capped percentage BUT ONLY if other classes that are using the line don't require more or less. it will divide that bandwidth accordingly. An example of this is that if the Fast class is set to 100% max and the Normal class is set to 50% at max. That's 150% and an unreal figure. Under no conditions would this actually allocate bandwidth using this method unless there are very small instances of low network activity. Still you would see something resembling a 89% Fast and 11% Normal or vice versa. That doesn't mean the Fast class is actually using all the download or upload pipe available, just helping the administrator to visualize the activity in the charts.

    When The Normal Class needs more than 50%, say 60%, Then the Fast Class lowers to 40% to make up for the difference and give 10% to the Normal Class at 60% because you only have 100% to give share!. More classes equal more pieces of the pie becoming smaller and less to go around. Again you will see more traffic in the Fast class than the Normal Class because the majority of transactions are small packets. That's why it's better to allow these out first to stop a "traffic jam" or unnecessary dropped packets and queuing. These small packets are usually latency sensitive and require less bandwidth while large packets need more bandwidth and less latency.

    *Note to Gamers*
    Understand tcp is the bandwidth hog not udp. Call of Duty for example only uses less than 512KB down/up for udp packets. TCP is the reason you need a recommended total of 2,000 Kilobits (2 Megabits) upload and at 2,000 Kilobits download to have a good gaming experience. UDP is communicating hitmarkers, voice, dns, etc. Tcp is transferring statistics, scores, visual content, game state etc. Without a structured system of QOS everything is thrown in a blender and causing more lag and erroneous gameplay. You don't need to allocate 100% of the pipe. The game chooses the host based on available upload speed at the start of each round regardless of any other conditions. If you must have multiple classifications set the small udp packets (udp ports) in higher priority with lower bandwidth and the large tcp packets (tcp ports) in less priority with higher bandwidth. Using the my suggested classes of Fast and Normal already does this without the need to set ports because it's based on the transfer size not ports, Hence it's "automatic"
     
    Last edited: Jan 27, 2017
    visceralpsyche likes this.
  13. visceralpsyche

    visceralpsyche Networkin' Nut Member

    Excellent explanation Jason :) Much appreciated!

    Here's what I've put together based on yours and @cloneman's settings. Will test over the next couple of days but a quick stress test already showed a nice improvement in responsiveness compared to previously.

    Hat tip to you both!

    [​IMG]

    [​IMG]

    Cheers,

    Paul :)
     
  14. visceralpsyche

    visceralpsyche Networkin' Nut Member

    One more question, with my particular setup - should I be setting BelowNorm as the default class, instead of Normal (since I'm copying @cloneman's three pronged strategy)?
     
  15. Jason Woringen

    Jason Woringen Network Newbie Member

    Yes below normal. Now if you want to live dangerously try 2 classes Small, Large both rates 50 Min 90 Max and 90% of your lowest measured upload and download speeds. Default Class Large. Small 0-1024, Large 1024+. Let that marinate in your minds.
     

    Attached Files:

    visceralpsyche likes this.
  16. visceralpsyche

    visceralpsyche Networkin' Nut Member

    Thanks again! BelowNorm now set as default class, and everything appears to be running well. Much obliged!
     
  17. Jason Woringen

    Jason Woringen Network Newbie Member

    You're very welcome! Keeps us posted on future updates either good or bad. Interesting to see the long term results.
     
    visceralpsyche likes this.
  18. cloneman

    cloneman Networkin' Nut Member

    hrmm, my mind is getting a big tangled up now, I guess Jason's approach was similar but with a "reversed" approach


    He puts everything in the bottom class, then pulls it up to FAST with a 0-512kb rule. What I usually do is put everything in the middle , then "push" it lower with a 512kb+ rule. It's a subtle difference. One that I'm sure people can resolve for themselves better than I can right now :confused:


    Here is my twist on the implementation of the technique discussed by Jason in this thread.

    Classification:
    Fast 0-512kb
    Default 512kb+
    BigXfer (Torrent Ports)


    [​IMG]





    Other notes: This approach allows quick web browsing and DNS resolution while limiting the congestion power of big file transfers, but it doesn't do much for gaming or voip, for which a 4 or 5 class system would be better.
     
    Last edited: Jan 28, 2017
    visceralpsyche likes this.
  19. Jason Woringen

    Jason Woringen Network Newbie Member

    My error cloneman, default class should be set to Fast/Small for my setup. Good eye.
     
    visceralpsyche likes this.
  20. Jason Woringen

    Jason Woringen Network Newbie Member

    These settings allow for A+ on ethernet and wireless. Note that ethernet and 5Ghz receive an A+ for bufferbloat, quality, and speed. Throughput is consistent and jitter is excellent for me at a consistent 4ms +/-. Min 60 Max 60 for ingress and egress produces the same results.

    Tomato Firmware 1.28.0000 -3.4-138 K26ARM USB AIO-64K
     

    Attached Files:

    Last edited: Feb 1, 2017
    gffmac likes this.
  21. gffmac

    gffmac Networkin' Nut Member

    Im currently running the attached rules, can anyone advise any tweaks. I would prefer to be able to use most of my download speed when I need it which is why I have 50%-100%.
    I want my xbox one to take priority no matter what else is happening, is there anything extra I can add to help out there?

    http://www.dslreports.com/speedtest bufferbloat test seems to be good with current settings. A/A+ most of the time.

    Also on the classification rules, is smaller better for responsiveness?

    Currently using a RT-AC68U 1000mhz overclock.

    Thank you.
     

    Attached Files:

  22. cloneman

    cloneman Networkin' Nut Member

    Add an xbox rule above the others, classified by your xbox's mac address.

    so you'd have

    10 - 100 (Xbox)
    40 - 100 (Fast)
    30 - 100 (Normal)

    Left side would add up to ~80 which is less than 100, so you're good.

    Optional:
    Misbehaving applications (i.e. steam downloads and torrents) will cause "Normal" downloads to interfere with other applications. To *somewhat* mitigate this, you can also introduce a "bulk" class or BigXfer class where you'll punish bad applications with less than 100%.
     
    gffmac likes this.
  23. txnative

    txnative Serious Server Member

    fixing bufferbloat and balancing your bandwidth using fq_codel should be enough as is without the need to overclock cpu, prioritize applications or packets but of course the guys are unaware of what kind of network you have and the size, torrents etc that are maybe needing prioritizations. Normally fq_codel handles and tames networks a great deal without the need to do all that, but if you were not using fq_codel in there then I would understand what you were trying to accomplish. A few years ago I had been using tomato K26rt-n for my e-series that didn't or still doesn't support fq_codel on the 2.6 kernel but using Tomato Qos was a excellent experience to learn and to have complete control, just have to tweak it, I kinda miss it.
     
  24. Jason Woringen

    Jason Woringen Network Newbie Member

    I'm now using openwrt again with the LEDE branch. I dusted off my old wndr3800ch from Charter cable and flashed that with openwrt. I then flashed my R7000 with stock firmware R7000-V1.0.7.10_1.2.3 . I set the R7000 as an AP only. My little wndr is now running with LEDE, Sqm scripts using cake as que discipline and piece of cake scripts. Ethtool set to ethtool -G eth1 rx 128 tx 128. To make changes persistent upon reboot, edit /etc/rc.local and comment "exit0" then add the ethtool parameters and save. Reboot and they should still be in effect. Best Setup and I'm completely happy. Inbound Qos 54067 (66000*1024*.80=54067) Outbound Qos (5000*1024*.80=4096). I did modify the sqm scripts to allow for some ports as high priority above all, only for the gamer in the house. 100% satisfied.
     
    Last edited: Feb 15, 2017
  25. txnative

    txnative Serious Server Member

    My couple of yrs that I had last used my old routers I then decided to use an old x86 pc that I started out with dd-wrt, worked perfectly but i wanted to do more with it then decided to go with openwrt x86 generic and was able to remove or install packages as desired and been using it ever since of i have those old routers(E3200/F7D4302) as ap with oem firmware for wifi coverage my son placed his R7000 running same version firmware 1.0.7.10_1.2.3 is also ap mode ethernet ports for all his ps4, tv, desktop wifi for that portion of the house I have an ethernet connection coming from linksys e3200 ap that is connected to a 5 port switch port that has my network connected and the frontier quantum gateway with the x86 router in front of everything. Currently I have been using luci-app-sqm, queue discipline fq_codel with simplest.qos, no priorities needed, unless I had a more complex network, that needs VoIp, torrents that need port forwarding etc. The tv service runs absolutely fine I have the triple bundle. I haven't used the piece of cake or other queue since the one's mentioned works perfect, although i'll have to do some testing when no one is here with cake, pie or the other queue scripts.
     
  26. Jason Woringen

    Jason Woringen Network Newbie Member

    I managed to squeeze up to 90% inbound and outbound bandwidth with good results. Turns out I was starving the receive/transmit buffers for the nic causing a crash and reboot with rx 16 tx 16; rx 64/23 tx 64/32 was still too low. Increased to rx 128 tx 64. I'm very please with Cake. It's almost there. Jim Gettys is driven! Openwrt is a masterpiece once you can understand it's infrastructure. Now if we could only get more manufacturers to play nice with open source drivers. Since my earlier post I've taken out the R7000. It's just too bloated. Great for a Lehman that only wants to web surf or do basic tasks. My little wndr3800ch with this setup is just wicked. everything jives now. fq_codel is like cotton as cake is to silk.
     
  27. Happi_singh

    Happi_singh New Member Member

    i have a comcast/xfinity 200 down and 12 up, i have tried all kind of setting with qos, i have tried Jason Woringen setting which will give A to A+ on the pc that is hooked with Ethernet, wifi gives C on bufferbloat. the main issue is even with ethernet hooked to xbox 360, cod bo2 still have hit detection issue or say latency issue, even with full green bars.
     
  28. cloneman

    cloneman Networkin' Nut Member

    QoS helps only to maintain reasonable latency performance during congestion. If you have latency / loss issues when your connection is not being stressed by a stream or a heavy transfer, QoS will not help you.

    Most likely, your ISP is not delivering good latency service. Perhaps you want something like Smokeping (https://www.dslreports.com/smokeping) to monitor your connection for 24 hours and see if its has problems. Or this https://www.dslreports.com/tools/pingtest for a live, realtime test (to be peformed during a latency event, e.g. during a game when you experience issues)
     
  29. Happi_singh

    Happi_singh New Member Member

    16/2/2017 04:11:04AM calculating grade
    16/2/2017 04:11:27AM grade A
    16/2/2017 04:11:57AM grade A
    16/2/2017 04:12:27AM grade A

    Location IP Min +PDV
    NSW, AUS 54.153.204.70 180 +11.5ms
    SC, Brazil 200.237.196.90 189 +14.7ms
    SP, Brazil 52.67.48.85 170 +3.6ms
    Manitoba, Canada 192.219.0.94 50 +12.7ms
    ON, Canada 206.248.155.54 42 +11.1ms
    QC, Canada 173.243.192.222 65 +9.4ms
    AMS, Netherlands 5.153.60.125 141 +16.1ms
    Germany, EU 95.172.92.166 166 +14.6ms
    Ireland, EU 54.154.207.39 158 +9.8ms
    London, UK 88.80.191.58 140 +11.9ms
    NL, EU 104.155.37.25 137 +14.7ms
    Zurich, EU 83.150.0.50 171 +10ms
    Mumbai, India 35.154.141.185 9999 +0ms
    Tokyo, Japan 54.65.9.39 129 +14.3ms
    Davao, Philippines 202.137.115.7 227 +10.1ms
    Singapore 54.179.167.120 225 +2.4ms
    Taiwan 104.155.201.41 145 +5.7ms
    CA, USA 64.140.161.58 23 +4.7ms
    CO, USA 72.5.102.138 34 +3.1ms
    DC, USA 65.79.226.210 81 +2.5ms
    DE, USA 162.151.17.198 66 +12.8ms
    FL, USA 99.24.18.30 65 +3.2ms
    GA, USA 104.153.104.126 52 +10.1ms
    IA, USA 104.154.69.141 47 +2.2ms
    IL, USA 162.248.92.123 47 +1.7ms
    IN, USA 99.24.18.74 53 +2.5ms
    KY, USA 199.193.180.23 62 +1.8ms
    LA, USA 162.248.93.162 31 +2ms
    MI, USA 68.85.49.234 68 +1.1ms
    MN, USA 64.90.65.50 64 +1.5ms
    MO, USA 142.54.161.173 47 +18.8ms
    NC, USA 99.24.18.90 77 +1.8ms
     
  30. Jason Woringen

    Jason Woringen Network Newbie Member

    Try testing with only 4 streams up, 4 streams down at dslreports.com/speedtest. You don't need 16 streams down and 6 up to saturate your connection. The flent test only uses between 4-8 streams both directions unless you have a large pipe and it won't saturate but unlikely at speeds 300MB down, 20 up or so. Sadly the older linux kernels didn't anticipate for bufferbloat which is why your scores are going to be limited. There's only so many things you can do to alleviate the pain with what is there. If Broadcomm and other manufacturers would release the full binaries without giving the tomato folks blobs of crud they could advance to newer kernels.

    Call of Duty has so many internal problems I don't know where to begin or even if I should. They must be using low budget servers and a trunk money to manage them. Frame Drops, high packet loss, inducing latency to better connections to compensate for someone with 250ms+ ping and an unstable bandwidth. Ever notice that when a player with less than 5 bars enters a lobby things get sloppy all around?

    They only things You can do are: Use a direct ethernet cable to you Xbox/Xbox One. If you can't run a cable purchase an ethernet powerline device. Many models to choose from direct plug them into a wall jack, not into a surge protector or anything else. DIRECT. Use AQM such as Cake, FQ_Codel, and even PIE if it's a last resort. Prioritize ports 88 UDP<Server/Membership Authentification, 500/4500 UDP< Nat Traversal, 3544 UDP<Teredo Tunnel, 3074 BOTH <Xbox Live, 3075 BOTH <Black Ops 3-Demonware; make sure they are port forwarded and a static ip address is assigned to your Xbox. Disable UPNP(doesn't play nice with Demonware). Don't bother with the other ports listed because they are either already whitelisted as basic ports for common use e.g. 80,500, Try reducing resolution from 1080p ( The game can't handle anything about 900p anyway) to 720p. Get a gaming monitor with 5ms or less. Otherwise it's out of your hands. I've read so many forums about Call of Duty and how they are supposed to have Dedicated Servers when in fact it's always been P2P for actual gameplay. You are always going to lag in that game because of it's bad design and continual lack of education toward the community and even the developers. Connections are based to each other, not to the server for action. The servers only purpose is to allow you to connect to other people and update stats, download content, majority of TCP connections. UDP is where the bread and butter is happening between hosts/clients in the lobby. I bet 2 guys from this forum could easily fix their problems in 1 week from Activision's end. The other side is gamers learning about how latency, jitter, and bandwidth interact on the lower scale.

    You shouldn't have to become a networking guru to play a simple game, but that's the world of technology we live in and it's only going forward. It is what it is unfortunately.
     
    cloneman likes this.
  31. cloneman

    cloneman Networkin' Nut Member

    Yeah, I guess its also possible the game servers aren't performing well. That's something neither you or your ISP can control.

    @Jason Woringen have you ever benchmarked QoS / fq_codel peformance when downloading from steam? I don't have access to anything faster than 50mbit, but steam has always created 4 - 10% packet loss in all my testing scenarios (unless I set it to 25mbit max, taking 50% off the table). Fq_codel doesn't "fix" it. They open 20 TCP connections to download a game, I think the problem might be a RWIN that is too large however.
     
  32. Jason Woringen

    Jason Woringen Network Newbie Member

    I think you mean the RWIN is too small? By reducing the bandwidth you are lowering the amount of data being sent before packet loss occurs. You may have to lower your MTU size.
     
  33. Jason Woringen

    Jason Woringen Network Newbie Member

  34. Jason Woringen

    Jason Woringen Network Newbie Member

    You need to start with disabling all wifi, and disconnect everything except your computer running only 1 browse tab and nothing else in the background. Clear all cookies to make sure you're signed into facebook, youtube, netflix, etc. These things run in the background and will affect the reading. disable QOS, then run 3-4 tests at dslreports.com. A Netperf/Iperf run will give more consistent and reliable results because testing at only 30 seconds isn't long enough. A time value to run would be closer to 60-90s, 90 or longer is the most accurate in my opinion to beat speedboost and other factors. Remember that your link doesn't download at the max rate continuously unless certain factors are tweaked and even then it will fluctuate.

    Take the LOWEST LOWEST LOWEST reported full values e.g. 65.3/5.4 and record that along with screenshots or pdf's of each sample. Again the site isn't great but it gives you a good picture of what's happening with your wan connection. Study the upload side only for the time being. Your upload affects your download. it's like a conversation; the less you speak, the less responses you receive. If everyone is speaking all the time and at once, it's hard to have a smooth conversation without stuttering and constant pausing. In the case of networking, this keeps the conversation flowing nicely. Enable QOS with 2 classifications, one for tcp/udp ports 1-65535 0-512KB and the second the same except 512+KB. Min 1 Max 100 on both ingress and egress. Enable Prioritize ACK and SYN. Try and tweak your upload till it's flawless. Note this may take 30 mins or more but it's worth it to get it right the first try. Start at 70% Upload and remember don't touch the download. You'll notice that your download will improve considerable by doing this. Fq_codel by default will subtract 5% +/- from the download and upload figures. If you enter 90% in the upload you're really at 85%. The same principal for the download. Don't be tempted to enter high figures for upload if it's not perfect, that's defeating the whole strategy behind AQM fighting bufferbloat. Once you have the upload tweaked to perfection, make minor adjustments to the download if any. THEN start creating priority classes and traffic shaping.

    Older linux kernels don't have nearly the adjustments for bufferbloat. make sure ecn,timestamps, dsack, and fsack, tcp slow start are enabled if possible. If ethtool was accessible with tomato, you could disable all offloading such as gro, gso, tso.
     
    gffmac likes this.
  35. Jason Woringen

    Jason Woringen Network Newbie Member

    These are my results. I have netperf results as well but not where I can upload them at the moment. Those results are far more accurate and much prettier numbers.
    I tried tweaking a thousand times and this method is the simple, tried and true way of making the magic happen. Notice my latency (ping) is identical which means job complete. Voip is crisp, Gaming awesome, Streaming flows and downloads with high bandwidth, very low latency. Web pages almost instant responsiveness. Mission Accomplished.
    Before > http://www.dslreports.com/speedtest/11260750
    After > http://www.dslreports.com/speedtest/11266225
     
    Last edited: Mar 1, 2017
    visceralpsyche likes this.
  36. gempotpot

    gempotpot New Member Member

    i try all of suggestion in this threads,
    and still got pocket loss in online game (dota2, league of legends)
    anyone can help me pls.
    im newbie in QOS...
     
  37. Sean B.

    Sean B. LI Guru Member

    Packet loss in online gaming is not always, one could even say rarely, an issue that is caused by your local configuration/network and therefor outside your ability to do anything about. Packet loss can occur anywhere a long the many many many many hops ( routers ) the packets go through between you and the game server. I suggest running a traceroute to the game server IP address as this will ( most of the time, but not always as these problems can be intermittent ) show you where along the route the packet loss happens or where latency jumps high.
     
    gempotpot likes this.
  38. txnative

    txnative Serious Server Member

    Can you tell us which model router and version firmware you are using, and may I ask if your pc hardware is updated all drivers and have a good monitor for gaming? To also tell us what Sean B asked about running tracerroute in Linux or tracert on windows of the server you are connecting to for a visual of problems with packet loss or high ping.
     
    gempotpot likes this.
  39. gempotpot

    gempotpot New Member Member

    thanks for the reply, and sory im not good in english...
    my router model is CW-5356U..
    in my speedtest without router or i direct in modem is 90mbps/DL and UL.
    i cant read that tracert, this is the first time i hear about tracert :(
     

    Attached Files:

  40. gempotpot

    gempotpot New Member Member

    i build my own internet cafe and i have 19computer...
    browsing is good, but if many costumer playing a online game...
    they always getting a problem with packet loss or high ping.
    pls help :(:(:(
     
  41. cloneman

    cloneman Networkin' Nut Member

    That router is listed at 300Mhz which would be too slow for use @ 90mbps with many users in an internet café scenario.

    It is the most likely cause of your issues... you need a dual core ARM 1000Mhz router such as the AC68U, or Edgerouter-X, or x86 based firewall. Edgerouter-X with smartqueue QoS may be sufficient to prevent packet loss or high ping.

    Once you have solved this, you may also want to consider installing a steam caching appliance server, as steam downloads and updates are very problematic, establishing 20 TCP connections and pounding your connection. Or, you can go to each computer and limit update downloads to 20mbps or something like that.
     
    gempotpot likes this.
  42. gempotpot

    gempotpot New Member Member

    i think router is not a problem, i have QOS rules,. and i use it for almost 2yrs without a problem.
    i remember that i copy a toastman build and i add a rules for online games..
    but now, the image can click :(
    last week i change change change settings, then reset. now i don't know how to get it back..
    :(:(:(:(
     

    Attached Files:

  43. Sean B.

    Sean B. LI Guru Member

    I have to agree with @cloneman . I'd most certainly investigate just how well that router is holding up under high demand like that before you end up chasing your tail trying to hunt down a problem. During high client count+ gaming loads, keep an eye on the "CPU usage" percentage via the routers web interface under Status->Overview. If you're seeing consistent usage of around 75 - 80%+ or frequent spikes up close to 100% then the routers ability to handle your traffic would be in question.

    As far as QoS goes, I can't really offer much help as I never use it. But your traceroutes give some odd information. There is one very clear spike, in the first traceroute run hop #11 shows a jump up to 208ms. That type of spike, if it occurs randomly yet frequently can very well cause a problem. What is very strange about it though is how the heck it's a local-only IP of 192.168.152.15 .. and the hop before that is another local-only IP of 192.168.150.30. These IPs showing up on the destination side ( the packets have exited your local-only network scope after the 2nd hop ) of a traceroute run to an outside ( internet ) location should not be possible. Could you do a traceroute to the IP address of 192.168.152.15 and post it please?
     
    gempotpot likes this.
  44. gempotpot

    gempotpot New Member Member

    any suggestion of cheapest router for tomato firmware?

    @Sean B. print screen of 192.168.152.15 tracroute.
     

    Attached Files:

  45. Sean B.

    Sean B. LI Guru Member

    Something's not configured correctly somewhere a long the line. Couple questions for you:

    1. What type of internet connection do you have? IE: Cable, DSL, Cel etc.

    2. For that connection, are you using ISP provided hardware? Such as a router/modem combo?

    3. Does streaming video from youtube or other sites become choppy or buffer excessively?

    I'd really suggest investigating what's going on with your network/connection before tossing money at a new router. Unless you want an upgrade or feel one is needed anyway.
     
  46. gempotpot

    gempotpot New Member Member

    1.cable, fiber connection.

    2.Combo (Modem router of ISP connected to CW-5356U router)

    3.sometimes (Server DNS address could not be found)

    ok i will check all my connection. :)
    Thx @Sean B.
     

    Attached Files:

  47. Sean B.

    Sean B. LI Guru Member

    Any chance your ISP modem/router has not been put into bridge mode?
     
  48. gempotpot

    gempotpot New Member Member

    i think theres no bridgemode in my modem router. :(
     
  49. Sean B.

    Sean B. LI Guru Member

    If your CW-5356U is configured to be the DHCP server/firewall/dns etc for your LAN ( the topology drawing you posted looks as though it is ) then this is a problem and would explain the local-only IP showing up on the traceroute where it shouldn't be. The router part of the modem/router and your CW router will conflict as the CW doesn't know it's WAN port is actually in the still local-only LAN of the modem/router.. treating the local-only IP it gets from the modem/router as if it's a global one. Problems with dns/routing/latency and especially port forwarding will happen.. how frequent will vary. What make/model is your ISP's router/modem?
     
    gempotpot likes this.
  50. gempotpot

    gempotpot New Member Member

    Name: EPON ONU
    Model: FH1001-6c0-42-b2
     

    Attached Files:

  51. ruggerof

    ruggerof LI Guru Member

    I don't understand the reason for traceroute the address 192.168.152.15 as your Tomato router subnet is 192.168.1.1/24
     
  52. ruggerof

    ruggerof LI Guru Member

    Apparently this modem-router has the option of bridge mode. http://www.cdr.pl/download/EP101Z_EPON_ONU_USER_MANUAL_v1.1.pdf

    Reading quickly the above manual I saw the below image, which indicates a double-nat already. I wonder if you are in a tripple-nat (does it even exist?). Can you please provide a screenshot of Status-WAN connection in your system?
     

    Attached Files:

    gempotpot likes this.
  53. gempotpot

    gempotpot New Member Member

    i think its a ISP problem, or wire..
    my wire(utp calble) is almost 7yrs...
     
  54. Sean B.

    Sean B. LI Guru Member

    Look at the first traceroute he posted, you should see why I asked for it, and also how I determined he's double natted.
     
  55. Sean B.

    Sean B. LI Guru Member

    I can guarantee you that your router/modem needs to be in bridge mode for proper network functionality. I'm not going to say it will fix every issue you have, but it is an indisputable misconfiguration. And at the very least will cause port-forwards on your CW router ( both manually configured ones or ones that are done via UPnP by games or other software ) to fail. I strongly urge you to correct the issue and see how the network responds.
     
    Sortec likes this.
  56. ruggerof

    ruggerof LI Guru Member

    Yes, I saw it after I have posted the question and indeed he is double.natted which can be seen in the 1st and 2nd hop of the traceroute(s) gempotpot has posted. I wonder though if the EPON ONU modem-router combo is already double-natted.
     
  57. gempotpot

    gempotpot New Member Member

     

    Attached Files:

  58. Sean B.

    Sean B. LI Guru Member

    Holy heck. In all honesty, it's absolutely amazing you can access the internet at all. That config is very, very.. not ok. If you want to set the router/modem to bridge mode and configure your Tomato router correctly, I'd be happy to help.. just let me know.
     
    gempotpot likes this.
  59. Nelbin Binag

    Nelbin Binag Reformed Router Member

    CW-5356u is your bottleneck. 300mhz! and thats a CDR-KING Rebrand!
    "EPON ONU modem" <-- im guessing youre on fiber (converge ict)

    Get a Netgear R7000 or any dualcore 1ghz that supports tomato.
     
    gempotpot likes this.
  60. gempotpot

    gempotpot New Member Member

    Yes i want help, but how?
     

Share This Page