1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Tomato Toastman's Releases

Discussion in 'Tomato Firmware' started by Toastman, Dec 18, 2011.

  1. kille72

    kille72 LI Guru Member

    I have BusyBox 1.25.1 in my ARM repo and I have no problem with TLS SNI support via openssl, your patch applied.

    Code:
    Jan 28 23:13:21 Asus user.info adblock: [2] downloading blacklist - https://adaway.org/hosts.txt
    Jan 28 23:13:21 Asus user.info adblock: ... [2] found 527 entries
     
  2. koitsu

    koitsu Network Guru Member

    Can you please provide me with all relevant commits that were involved in upgrading from 1.23.2 to 1.25.1? It isn't as easy as just untarring the tarball, I assure you (I've had a convo with Toastman about this privately).
     
  3. M_ars

    M_ars Network Guru Member

    did test the RT-AC-patch with a custom-511-build on my RT-N16 - no problems so far. Everything i tried did work, erase nvram, restore/backup, setting everything up from scratch, ...

    I can not test ARM or ARM7
     
  4. ruggerof

    ruggerof LI Guru Member

    I found a possible (minor) bug.

    Router: R7000
    Firmware: Tomato Firmware v1.28.9008 Toastman-ARM K26ARM USB VPN-64K

    Under Administration, Logging, Syslog, I checked "NTP" events to be logged but nothing is shown in the log.

    This behaviour doesn't show in my AC66U running Toastman v1.28.0510 MIPSR2Toastman-RT-AC K26AC USB VPN nor in my AC68U running Shibby 136 AIO.
     
  5. M_ars

    M_ars Network Guru Member

    i did make two patches for Toastman branch mips (RT-AC) & arm to increase the security for samba3 a little bit :)
    files/patches are attached if someone is interested


    https://www.samba.org/samba/security/CVE-2012-0870.html
    https://www.samba.org/samba/security/CVE-2015-5252.html

    - file 0001*.patch : security update for samba, CVE-2012-0870; Subject: Remote code execution vulnerability in smbd
    - file 0002*.patch : about page - cosmetic: correct dnscrypt-proxy build
    - file 0003*.patch : security update for samba, CVE-2015-5252; Subject: Insufficient symlink verification in smbd.


    best regards
    M_ars
     

    Attached Files:

    William Clark, Elfew and kille72 like this.
  6. azdps

    azdps LI Guru Member

    kille72's busybox commit history has everything you need. shibby made the initial upgrade to 1.25.0.

    https://bitbucket.org/kille72/tomato-arm-kille72/commits/all?search=busybox

    Not sure if these patches from busybox are already patched against the source code kille72 is using or not so I've listed them below. Not even sure they are needed. Official patches for busybox v1.25.0 and v1.25.1:

    https://busybox.net/downloads/fixes-1.25.0/
    https://busybox.net/downloads/fixes-1.25.1/
     
    Last edited: Jan 29, 2017
  7. ambiance

    ambiance Serious Server Member

    @koitsu It sounds like someone will have to break out the Advil and/or Tylenol for the Frankenjob that will entail. Also, I'll try to remember to disclose all the relevant details next time.
     
  8. kille72

    kille72 LI Guru Member

    Andre was first with the 1.25.1:
    https://bitbucket.org/AndreDVJ/advancedtomato-arm/commits/all?search=busybox
     
  9. azdps

    azdps LI Guru Member

    Last edited: Jan 30, 2017
    Edrikk and kille72 like this.
  10. William Clark

    William Clark Reformed Router Member

    hey everyone,

    so I have a issue with Router time not getting updated.
    Don't know, why sometimes the router shows "Router Time -> Not Available"
    even though I have set the DNS server to 208.67.222.222!
    And I can ping "0.europe.pool.ntp.org" from Tools -> Ping.

    Round-Trip: 101.730 min, 102.817 avg, 103.933 max (ms)
    Packets: 5 transmitted, 5 received, 0% lost

    I need, router time gets updated, because I have to restart my router at a certain time everyday.
    Thanks
     
  11. Toink

    Toink Network Guru Member

    Have you tried selecting the NTP server to "custom" then use these on the 3 fields?
    "ntp-s1.cise.ufl.edu
    time01.mel.optusnet.com.au
    2.europe.pool.ntp.org"

    Just make sure you have selected Auto Update time to your liking. I set mine to 'every hour' and make sure you are in your correct time zone.
     
    William Clark likes this.
  12. koitsu

    koitsu Network Guru Member

    Telnet/SSH into the router and run the command ntpsync and provide here the output.

    Odds are you haven't set up Basic -> Time correctly, or your network configuration (upstream from you, or your topology) is doing something bad with outbound packets destined to UDP port 123. Your ISP, for example, may be limiting NTP requests (ex. they may run their own NTP servers and thus require you to use them instead of things on the Internet). We simply don't know.
     
    William Clark likes this.
  13. William Clark

    William Clark Reformed Router Member

    Thanks a lot for your reply.
    I did what you said, Enabled Custom NTP time server, but still "Router Time Not Available". :(
    by the way, should it get updated immediately, correct?
    Oh, and Auto Update Time is on "Every Hour"
    and Trigger Connect On Demand is checked.
    And my time zone is not available in Tomato, so I had to add the custom time zone.
    Thanks.

    Thanks a lot for your reply.
    When I SSH-ed into the router, I get the following:
    -sh: ntpsync: not found

    I don't think my ISP is blocking NTP requests. Because sometimes it gets updated immediately and sometimes it doesn't, but now it's been 2 days and the time did not get updated (Router Time -> Not Available), and I needed the router to restart itself at a certain time, but unfortunately because of this problem, it did not :(.

    Thanks
     
  14. ruggerof

    ruggerof LI Guru Member

    Just to shed a light on this.
    ntpsync exists in my AC66U running Toastman v1.28.0510 MIPSR2Toastman-RT-AC K26AC USB VPN
    ntpsync exists in my AC68U running Shibby 136 K26ARM USB AIO-64K
    ntpsync does NOT exist in my R7000 running Toastman v1.28.9008 Toastman-ARM K26ARM USB VPN-64K

    All of them seem to sync the time with my Fritzbox router (it can act as a ntp server), the only difference is that my R7000 does not include NTP related action in the log as I mentioned here.
     
    William Clark likes this.
  15. William Clark

    William Clark Reformed Router Member

    Thanks man, yeah, my router is R7000!
    is there anyway, which I can install ntpsync on my router?

    Thanks.
     
  16. ruggerof

    ruggerof LI Guru Member

    I don't know.

    I think that there is possibly a bug. Reading @koitsu thread it seems that a cronjob should trigger the command ntpsync. Comparing my three routers "cru l" command the cronjob to sync is not present in my R7000.

    Code:
    AC68U (Shibby 136):
    0 0 */1 * * /usr/sbin/webmon_bkp #webmon_bkp#
    59 * * * * /usr/sbin/webmon_bkp hourly #webmon_tmp#
    0 5 * * 0,1,2,3,4,5,6 sched sch_c2 #sch_c2#
    */1 * * * 0,1,2,3,4,5,6 sched sch_c3 #sch_c3#
    0 12 * * 0,1,2,3,4,5,6 sched sch_c5 #sch_c5#
    28 10 * * * /usr/sbin/tomatoanon checkver #checkver#
    28 10 * * 2 /usr/sbin/tomatoanon #anonupdate#
    41 22,23,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21 * * * ntpsync --cron #ntpsync#
    0 */1 * * * logger -p syslog.info -- -- MARK -- #syslogdmark#
    */15 * * * * rcheck --cron #rcheck#
    56 10 30 1 * sched sch_c1 #sch_c1#
    40 0 20 2 * ddns-update 0 force #ddnsf0#
    9 * * * * ddns-update 0 #ddns0#
    
    AC66U (Toastman):
    0 */1 * * * logger -p syslog.info -- -- MARK -- #syslogdmark#
    */1 * * * 0,1,2,3,4,5,6 sched sch_c1 #sch_c1#
    */5 * * * * /cifs1/stealthMode perm_on #stealthmode#
    50 21,22,23,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20 * * * ntpsync --cron #ntpsync#
    
    R7000 (Toastman)
    0 */1 * * * logger -p syslog.info -- -- MARK -- #syslogdmark#
    */1 * * * 0,1,2,3,4,5,6 sched sch_c1 #sch_c1#

    EDIT: Comparing ARMv7 and MIPSR2 Toastman. Output of "find / -name ntp*

    Code:
    AC66U
    
    /bin/ntpc
    /bin/ntpstep
    /bin/ntpsync
    /rom/etc/l7-protocols/ntp.pat
    /usr/sbin/ntp2ip
    /usr/sbin/ntpd
    
    
    R7000:
    
    /rom/etc/l7-protocols/ntp.pat
    /usr/sbin/ntpclient
    /usr/sbin/ntpd
    
     
    Last edited: Jan 30, 2017
    William Clark likes this.
  17. William Clark

    William Clark Reformed Router Member

    When there is no ntpsync in R7000 and no cronjob to trigger the ntpsync command, then how is it possible, that sometimes, my router time gets updated and sometimes it doesn't?
    It means, that there is some bug with R7000 routers?

    Thanks.
     
  18. koitsu

    koitsu Network Guru Member

    Let's not start making blind assumptions and claims, re: "bug with R7000 routers". Let's figure out what is going on logically. Do not flail arms.

    MIPS and ARM differ. Shibby and Toastman differ. And yes, sometimes there are different features/things that vary per model (this is more rare, but it's still valid). I don't want to sit around and "compare a bunch of firmwares" if I don't have to. This thread is about Toastman, thus, let's discuss Toastman.

    For NTP, yes, there is a difference between MIPS and ARM. There is no difference between ARM models when it comes to NTP, however. I have looked at the source code (Toastman-ARM branch) and verified this.

    Foremost: ntpd is not relevant to this discussion. It is Busybox ntpd, and built because I asked for it to be built back in May 2016. There is a thread I have from a few days discussing how/why this is important and relevant -- find it and read it if you care. Point is: ntpd does not get used on MIPS or ARM as of this writing. I'm working on it, it's going to take me a very very long time.

    MIPS has a cronjob that runs ntpsync at an interval chosen per Basic -> Time, defaulting to 4 hours. Under the hood, this effectively runs ntpc (Busybox NTP client) via cron.

    ARM has no cronjob. Instead, Tomato rc/init itself runs ntpclient -h {server} -i 3 -l -s at boot up. More specifically, the equivalent of service ntpc start is issued, which then issues that ntpclient command. Internally the start_ntpc() function (inside of rc/init) is used. There are also some Save operations in the GUI that will trigger start_ntpc(), as well as (I think) some bits like the WAN going down then coming up. I tried recently documenting what all the services and functions do (since many call others); that's in some other thread of mine, but it varies based on what firmware you use and what features.

    {server} is the NVRAM variable ntp_server. More on that in a moment -- it's relevant, trust me.

    ntpclient is an incredibly stupid/wonky/bizarre/weird NTP client I have never seen before in my life, written by Larry Doolittle. I literally have no idea where someone found this code. I have no faith in it. The website for this client is long dead/gone (cool/awesome!). So let's talk about the ntpclient flags:

    The -h {server} flag specifies an NTP server. I've looked at the code: only a single server is supported. The Tomato code uses the word "servers" to imply more-than-one, but ntpclient only supports one server. But again, more on this in a moment.

    The -i 3 flag makes ntpclient "poll every 3 seconds". But with the -s flag below, this essentially acts as a timeout.

    The -l flag "locks" the local system clock using the adjtimex(2) syscall.

    Finally, the -s flag represents "simple clock set": it effectively is the same as doing -c 1. The -c flag defines the number of times ntpclient, at the -i interval specified, to attempt NTP sync. The default value for -c is 0, which means run infinitely, so -s (thus -c 1) essentially says "do this once and once only".

    It is important to understand that ntpclient is not a daemon. It's essentially a "one-time-use" program; it's like ntpc (in theory) like on MIPS.

    Now, to talk about the -h argument a bit more: Tomato's start_ntpc() function literally does this:

    Code:
    1512         if (nvram_get_int("ntp_updates") >= 0) {
    1513                 strcpy(servers, nvram_safe_get("ntp_server"));
    1514                 xstart("ntpclient", "-h", servers, "-i", "3", "-l", "-s");
    1515         }
    
    But as I said, ntpclient as I said only supports one argument to -h. You cannot specify -h more than once either (well, you can, but whatever one comes last is what will get used). So, if in Basic -> Time you have 3 servers listed, the following will get run:

    Code:
    ntpclient -h server1 server2 server3 -i 3 -l -s
    
    server2 and server3 in this context serve no purpose; they will never be parsed/used/cared about. Only server1 is relevant. So, this is one example of where the Tomato code in ARM is wrong. To prove my point, here are some examples:

    Code:
    root@unknown:/tmp/home/root# ntpclient -h pool.ntp.org blah foo -i 3 -l -s
    setup_receive:: bind...
    setup_transmit:: connect...
    
    send packet OK!
    Recvfrom pack_len= 48, incoming= 1500
    call udp_handle
    UDP_handle: 48
    [ntpclient] set time to 1485778619.651790
    cat: can't open '/var/run/ntp.pid': No such file or directory
    sh: you need to specify whom to kill
    
    Cool/awesome!

    Yes, the cat and sh stuff is in the ntpclient code -- this looks to be a Tomato hack someone put in place in the ntpclient code. It's a travesty. Here it is:

    Code:
    532                 if (!primary_loop(usd, probe_count, cycle_time)) {
    533                         nvram_set("ntp_ready", "1");
    534                         doSystem("kill -SIGTSTP `cat %s`", "/var/run/ntp.pid");
    535                         close(usd);
    536                         break;
    537                 }
    
    So if there is no /var/run/ntp.pid (don't ask me why someone picked that filename rather than ntpclient.pid), you're going to see these errors. This is because Tomato's rc/init function xstart() will actually make a pid file (if I remember right -- could be wrong though, been a bit). I really don't see the point of the pidfile in this case.

    You can also see, however, that NVRAM variable ntp_ready gets set to 1 when things are done successfully. Guess what uses that NVRAM variable? Absolutely nothing. Cool/awesome! Purposeless waste!

    But wait, there's more!

    ntpclient will effectively do a DNS lookup on the NTP server specified. If that DNS lookup fails, the client simply bails out and will not retry. Here's proof:

    Code:
    root@unknown:/tmp/home/root# ntpclient -h fakehostname.derp -i 3 -l -s
    setup_receive:: bind...
    fakehostname.derp: Unknown host
    
    Even without -s and with -c 0 (i.e. probe forever) it doesn't retry. Cool/awesome!

    So now with all of the above in mind, I want you to think about this scenario:

    Upon reboot of your router, the WAN isn't functional yet (I'm not talking about "if it's up", I'm talking about whether or not packet flow is working). Tomato rc/init starts ntpclient, which tries to do a DNS lookup on the NTP server you specified, but DNS isn't functional yet due to the WAN situation (timing issue). ntpclient then bails out and never sync the clock.

    There is no "wait for packets to work on the WAN" support in Tomato natively. There are ways to do this in shell scripts (I wrote something like this for Entware-ng, and I wrote similar for FreeBSD several years ago which everyone uses to this day), but not natively. It's complicated to do because everyone's systems and configurations are different, you can't just say something like "hey man, ping this magic IP address and if it works, yeah!" because this essentially deadlocks the router until it would get an ICMP response. Trust me: it's complicated. It sounds easy, but it isn't. This is a problem that "plagues" every operating system on earth, it's not Tomato-specific in the least.

    And finally, I saved the best thing about ntpclient for last: here we have good evidence of "weird crap going on with this NTP client" and use of pool.ntp.org. I have seen this happen outside of Tomato as well -- which is why I don't use ntp.org for NTP servers, because I simply cannot trust all their servers and DNS to work correctly all the time:

    Code:
    root@unknown:/tmp/home/root# ntpclient -h pool.ntp.org -i 3 -l -s
    setup_receive:: bind...
    setup_transmit:: connect...
    
    root@unknown:/tmp/home/root#
    
    What happened here? Absolutely nothing! I can tell you that the client "sat there" for about 3 seconds -- no DNS failure, just flat out issued a UDP packet (or did it? I see no "send packet OK!" message, so...?!?), sat around for 3 seconds, then mysteriously ended. Good stability! Cool/awesome!

    I then did this immediately after -- yes, the exact same command:

    Code:
    root@unknown:/tmp/home/root# ntpclient -h pool.ntp.org -i 3 -l -s
    setup_receive:: bind...
    setup_transmit:: connect...
    
    send packet OK!
    Recvfrom pack_len= 48, incoming= 1500
    call udp_handle
    UDP_handle: 48
    [ntpclient] set time to 1485779042.527503
    cat: can't open '/var/run/ntp.pid': No such file or directory
    sh: you need to specify whom to kill
    
    Worked that time! High quality reliability we have here. Also, lack of syslog logging messages is great too, that way users have no way of knowing if something went wrong and if so when/why; instead, just spit everything to stdout/stderr. Cool/awesome design for embedded devices!

    So like I said: I do not trust this ntpclient program, and the model/methodology used in ARM is broken anyway. Whoever implemented this did so hastily and wasn't fully thinking it through. It should be thrown out.

    If you want a workaround, there are a few, but none of them are going to deal with the situation "properly". The "proper" solution is to use Busybox ntpd from Busybox 1.26.0 or newer, which runs as a daemon, supports more than one server, and will re-try even on DNS lookup failures. As I said above: I have a thread about this, and doing the work for it is tedious. (I began today, and I spent almost 4 full hours looking at **one of several Tomato modifications** to the Busybox code. JUST ONE OF SEVERAL! Tomato has heavily modified Busybox code, so upgrading Busybox is a serious PITA)

    If I had to give a recommendation for ARM workarounds right now, it would be this:

    1. Use an NTP server you know is reliable as possible. I can't tell you how to do this; it's up to you to figure it out.
    2. In the NTP server list in Tomato, only specify an IP address, to avoid chances of DNS failure on boot (timing).
    3. In the NTP server list in Tomato, only specify 1 NTP server; the rest don't get used right now anyway on ARM.
    4. Add a cronjob that syncs the clock periodically (similar to how MIPS does it). You could use something like this in Scripts -> Init specifically so that it gets run on boot:

    Code:
    cru a ntpclient-workaround "0 */4 * * * service ntpc restart"
    
    The placement of double-quotes and where they are is important.

    This will essentially run ntpclient every 4 hours at xx:00 (ex. 00:00, 04:00, 08:00, etc.). If using public NTP servers, DO NOT set this to anything more than once an hour (*/1) else you will get banned or make an NTP server admin very angry.

    You can also run that command from the CLI manually (once only please, although repeated runs of it will not add duplicates) if you don't want to reboot.

    Hope this sheds light on the abysmal state of all of this.
     
    Last edited: Jan 30, 2017
    William Clark and kille72 like this.
  19. RMerlin

    RMerlin Network Guru Member

    azdps, M_ars and William Clark like this.
  20. William Clark

    William Clark Reformed Router Member

    @koitsu, thanks a lot man for your great reply. At first I understand almost nothing but after reading it for the second time, I understand some of it :D.
    When I issued the command: ntpclient -h pool.ntp.org -i 3 -l -s the time got updated almost
    immediately. Thanks.
    That was pretty interesting, that the second and third server serve no purpose!
    I setup a NTP server on my CentOS server, and pasted my server's IP into the first server field, and issued the command ntpclient -h myServerIP -i 3 -l -s and everything works good.
    So I think, having my own NTP is the best method.
    And thanks a lot for the cron job, pasted the cron job exactly like you typed in "Administration -> Scripts -> Init".

    Thanks
     
  21. azdps

    azdps LI Guru Member

    This is how commits should look like. Explains the purpose behind each change so later on when a new developer looks at the code it actually makes sense.
     
  22. koitsu

    koitsu Network Guru Member

    While the commits look fine, and the effort done there is 100% fantastic, the reverse-engineering effort to figure out the Tomato changes is still required regardless. The one I spent 4+ hours on, for example, isn't the same in Merlin as it is in Tomato. Screw it, I'll talk about it:

    The issue is with the utilities e2fsck, e2label, fsck*, mke2fs*, and tune2fs. Busybox at one point -- up until 1.24.0 -- used to include a kind of "hacked up/ripped out" version of these utilities (taken from e2fsprogs) located in busybox/e2fsprogs/old_e2fsprogs. However, as of 1.24.0, all that got deleted. Specifically, mke2fs* and e2fsck (and the fsck* equivalents) are gone, and so is blkid.

    Tomato "backports" the utilities that were in that directory. But guess what? The code Tomato uses differs from the Busybox code too! Yes, you heard me correctly, the code in the Busybox e2fsprogs utilities, which are no longer supported/included with Busybox, are different than the ones used in Tomato. There's about 200 lines of changes (I understand most, but not all).

    For these utilities to get built, the official Busybox Makefile also has to be tweaked.

    But it gets worse.

    The proper solution, as is implemented in Merlin and any normal/sane Linux distro, is to simply use e2fsprogs. That is also what the Busybox folks advocate (and have since roughly 2008 or so, I found posts from Denys on the busybox mailing list abut it).

    Tomato has e2fsprogs in its repo (see router/e2fsprogs), but it's only built/used if NFS support is enabled during compile-time. In Toastman-ARM, this is not the case for any targets -- I checked (I haven't looked at MIPS) -- but I would bet money there are targets in Shibby that include NFS, so I can't go revamping the code (just keep reading).

    But with NFS=y, the e2fsprogs utilities aren't actually installed -- the whole compile happens and binaries generated, but none of the utilities are used (there is no e2fsprogs-install target). Instead, e2fsprogs is compiled because nfs-utils relies on some code/bits from it.

    So to be clear/rephrase: even if NFS=y during Tomato compile-time, the utilities/binaries from e2fsprogs aren't used -- the old ancient hacked up utils from old Busybox are (and use different code from the Busybox versions). All e2fsprogs is used for is to make nfs-utils compile/happy.

    The solution seems pretty simple, right? Stop using this Busybox hacked up crap and use e2fsprogs officially. Have I tried this? Yes; my router/Makefile modifications look very similar to that of Merlin.

    But guess what the problem is? The e2fsprogs binaries (and libraries) are incredibly fat. Asssuming shared libraries are built (they aren't by default, and Tomato even with NFS=y doesn't enable them -- don't ask, it looks like an oversight), the libraries come to around 400KBytes, and the utilities themselves add another 200-300KBytes.

    I am absolutely certain this is why someone (Shibby, from the look of it) continues to "backport" the old Busybox versions: because the code is ridiculously smaller. You can't tell because of how everything in busybox gets compiled into a single binary then symlinks are used to represent argv[0], but it's no where near 600-700KBytes. It's probably more like 30KBytes.

    So, thusly, I reached an impasse. This is where the Tomato and Merlin projects diverge (welcome to why Tomato supporting 9 billion models of routers, IMO, is a nightmare and unfeasible). A 700KByte increase in firmware size is huge. Tomato still has lots of people using routers with 8MByte flash (and occasionally I see 4MByte flash people show up, sigh).

    Someone might ask this: "so just don't build the e2fsprogs stuff on smaller models of routers!" Yeah, sorry, it doesn't work like that. Said utilities are very important, particularly for any device that has USB support (and yes, there are 8MByte flash routers with USB). So like I said: impasse.

    This is just one of about 4000 lines of differences between stock Busybox and Tomato's Busybox.
     
    misuercarriere, Justio, azdps and 2 others like this.
  23. RMerlin

    RMerlin Network Guru Member

    Personally, I ditched that old junk because I doubted it would be able to properly deal with multi-terabytes partitions (tho even with the 512 MB RAM of an RT-AC88U, large partitions still fail to run efs2ck due to lacking sufficient memory to handle the large tables), GPT or 4K sector formatting. That was about the same time I started backporting GPT and Advanced Formating support to Busybox (mostly fsdisk at the time), until I said "screw that" and upgraded to what was the latest at the time - 1.20 or 1.21 (can't remember exact version), and didn't bother trying to get the old fs tools to build under the newer busybox, and went with the real package.

    There comes a point where you have to decide if you want to keep supporting obsolete models at the detriment of the newer models. I chose not to bother with the RT-N12 (small RAM and flash), and dropped the RT-N16 (not enough nvram) a few years later.
     
    azdps and William Clark like this.
  24. azdps

    azdps LI Guru Member

    koitsu I would like to see you create a new tomato repo with good coding practices. Eliminating a lot of unnecessary code hacking trying to keep binary and code size at a minimum for older routers. RMerlin makes a great point that there has to be a point in time where older router models become obsolete and no longer compatible. I would rather see more secure code then back-ported hacked code potentially introducing exploits. Not to say this is the case. I'm not challenging you by any means but have been watching you for years now. I agree with about 96% of what you say, and the coding horror stage that tomato is currently at.

    At this point I don't trust tomato or Asus code and only willing to use it for access point purposes. I would feel more comfortable with someone like you chipping away at all the trash one step at a time making tomato a more stable. I look forward to following your tomato repo.
     
    William Clark and gffmac like this.
  25. koitsu

    koitsu Network Guru Member

    Are you volunteering me for something I haven't agreed to do? Little bit rude. :p

    The only reason I have a fork in GitHub right now is because I don't have repo.or.cz access, either to a personal account on there, or to the Tomato repo. I definitely don't want the latter, and I don't want the former for the reasons outlined in the other thread (re: if Tomato repo should be at GitHub or somewhere else). I use my own repo as a way to "make changes" that I then link to Toastman in a PM (I give him the branch name, the commit(s), and the branch it was based on, and I try hard to do it for Toastman-RT-AC, Toastman-ARM, and Toastman-ARM7 where applicable). Don't think that just because I have a fork in GitHub that it means something bigger; that's really all I use it for. Anyway, focusing on the matter...

    It's easier said than done for several reasons. This is a matter of opinion, but at least with OSS, I've been part of major projects for over 2 decades (Linux and FreeBSD are the main two). I've seen a lot of OSS projects fail, and often for the same reasons (history commonly repeats itself in OSS):

    1. SPoF: with open-source projects, it's generally not a good idea to have a single person "responsible" for soemthing; if that person stops working on the project -- or in extreme cases, dies -- then there's nobody else to picks up the slack. Just because the source code is "out there on the Internet" doesn't mean the majority of people have the skill set to pick up where it left off (i.e. most end-users do not program, and those who do often do not program well; I'm not a professional programmer (by career), for example, I'm a systems administrator).

    2. Interest and time: something of this nature requires the responsible party/individual to have personal interest in it. Meaning: they must enjoy what they're doing (on some level) for it to feel rewarding, otherwise what you have isn't a hobby but a job. Something like a firmware requires a lot of time, and a very diverse skill set (for example, wireless is not something I'm good at, and I consider it a "best-effort" protocol, i.e. most of my own things are wired and I put very little focus on wireless. I'm very different from the majority). Just because I've been unemployed since May 2015 doesn't mean I have the time or interest to do such a thing. It's a humongous undertaking, and I must stress the word humongous. If you asked me if I found doing commits/stuff on Tomato enjoyable as a hobby, my answer would be no, I don't. It's more tedious than it is enjoyable.

    3. Subject becomes political (Tomato-wise) very quickly. I'd rather not go into depth on this one given what pieces I know of the history, but... hmm... how to approach this. Let's just say that another major Tomato fork would be, in my opinion, detrimental to the project. Having tons of forks is what (essentially) got us into this situation in the first place.

    I'm old enough to know when I see shambling or chaotic infrastructure (this applies to OSS as much as it does city/state infrastructures!), and I'm pretty vocal when I see it. That doesn't necessarily mean I'm willing to go fix it all (see above). The historic rebuttal I've given when I say such things (like when I chastised the Apache project many many years ago) was "send patches or STFU", which I've always thought was hilarious. I've never believed in the "send patches" OSS mentality, I've always believed in "let's work together, understand what's broken, and fix it properly".

    I'd have more to say, but I really can't be arsed to. I think that's enough to ponder over for now.
     
  26. azdps

    azdps LI Guru Member

    Fine! My recruiting skills aren't what they use to be. Like I said, I agree with what you say about 49% of the time.
     
    ambiance and gffmac like this.
  27. Jacques

    Jacques LI Guru Member

    Hi Toastman,
    I'm using Toastman tomato-K26USB-NVRAM64K-1.28.0511.5MIPSR2Toastman-RT-N-VPN on the RT-N66U
    My LAN range is 192.168.1.40-99 (static dhcp).
    IP Traffic in Last 24 Hours and Real-Time show the strange not programmed IP 192.168.1.11 192.168.1.12 192.168.1.14 192.168.1.16 192.168.1.23 192.168.1.27 192.168.1.101 192.168.1.224 without any traffic.
    Capture.PNG
     
    Last edited: Feb 7, 2017
  28. Jacques

    Jacques LI Guru Member

    Hi Toastman,
    Appeared two new unwanted ip 192.168.1.2, 192.168.1.4, 192.168.1.96

    Capture3.PNG
     
    Last edited: Feb 7, 2017
  29. srouquette

    srouquette Network Guru Member

    could be static IP, instead of relying on the DHCP.
     
  30. Jacques

    Jacques LI Guru Member

    Hi Toastman,
     
    Last edited: Feb 7, 2017
  31. M_ars

    M_ars Network Guru Member

    Can you provide more infos about your setup?
     
  32. Jacques

    Jacques LI Guru Member

    STATIC DHCP/ARP/IPT
    Capture5.PNG


    and in WIRELES CLIENT FILTER with PERMIT ONLY THE FOLOWING CLIENTS assignet by STATIC DHCP/ARP/IPT
     
  33. Dutch87

    Dutch87 Networkin' Nut Member

    Same thing here, no idea why... v1.28.9008 Toastman-ARM K26ARM USB VPN-64K
     
  34. Beast

    Beast Network Guru Member

    I get the same IP's as well. Tomato Firmware v1.28.7511 MIPSR2Toastman-RT K26 USB VPN

    Built on Fri, 20 Jan 2017 21:56:51 +0700 On RT-N16.

    192.168.1.2, 192.168.1.4, 192.168.1.69 <---- mine is that.
     
  35. M_ars

    M_ars Network Guru Member

    we have one commit (ae2a1a0ec520446c14954a8ce508d822e0dbc1a1) from February 2016
    "Fix phantom traffic data in IPTraffic"
    http://repo.or.cz/tomato.git/commit/ae2a1a0ec520446c14954a8ce508d822e0dbc1a1
    ==> but i think that was only about pseudo traffic/data not phantom IPs

    At "CC" git http://repo.or.cz/tomato-rt-n10.git there is also an interesting commit
    http://repo.or.cz/tomato-rt-n10.git/commit/ac11ba2312aa7a596a142bf4bcfa53c4deb783df
    cstats: Hide hosts with no traffic on IP Traffic "Last 24 Hours"
    ==> that could help (not tested, verified, ... will have a look :) )

    Code:
    --- a/release/src/router/cstats/cstats.c
    +++ b/release/src/router/cstats/cstats.c
    @@ -385,12 +385,25 @@ static void load(int new) {
            }
     }
     
    +int speed_empty(Node *node) {
    +       int flag, max;
    +       uint64_t *ptr;
    +       max = MAX_NSPEED * MAX_COUNTER;
    +       ptr = (uint64_t *) node->speed;
    +       flag = 0;
    +       while (!flag && max--)
    +               flag |= *ptr++;
    +       return !flag;
    +}
    +
     void Node_print_speedjs(Node *self, void *t) {
            int j, k, p;
            uint64_t total, tmax;
            uint64_t n;
            char c;
     
    +       if (speed_empty(self)) return;
    +
            node_print_mode_t *info = (node_print_mode_t *)t;
     
            fprintf(info->stream, "%s'%s': {\n", info->kn ? " },\n" : "", self->ipaddr);

    Edit:
    "int flag" should be "uint64_t flag" because of the bit operation and ptr (or node->speed) being a 64 bit value. Convert a long value to int (without cast) - I dont think that is an good idea
    Code:
    flag |= *ptr++;
    Edit 2:
    patch/code with some cosmetic - will try a testbuild tomorrow
    Code:
    +++ b/release/src/router/cstats/cstats.c
    @@ -385,12 +385,27 @@ static void load(int new) {
         }
     }
     
    +int speed_empty(Node *node) {
    +    int max;
    +    uint64_t flag;
    +    uint64_t *ptr;
    +    max = MAX_NSPEED * MAX_COUNTER;
    +    flag = 0;
    +    ptr = (uint64_t *) node->speed;
    +    while (!flag && max--){
    +        flag |= *ptr++;
    +    }
    +    return !flag;
    +}
    +
     void Node_print_speedjs(Node *self, void *t) {
         int j, k, p;
         uint64_t total, tmax;
         uint64_t n;
         char c;
     
    +    if (speed_empty(self)) return;
    +
         node_print_mode_t *info = (node_print_mode_t *)t;
     
         fprintf(info->stream, "%s'%s': {\n", info->kn ? " },\n" : "", self->ipaddr);

     
    Last edited: Feb 8, 2017
    William Clark likes this.
  36. Jacques

    Jacques LI Guru Member

    I noticed a problem with the iPad already a week are 1500 km from my home and are still visible.
    Other WiFi devices behave without a problem.

    Capture6.PNG
     
  37. koitsu

    koitsu Network Guru Member

    @M_ars I spent some time looking at this code (about 3-4 hours) and creating makeshift version for testing some theories I had. Your changing of flag to uint64_t is what got me curious.

    I'm thinking the author of this code wrote it the way he/she did purely for bragging rights -- or to give him/her benefit of the doubt, maybe out of pure habit (and I would love to know the basis of those habits). There are several things in this tiny function that are not immediately obvious to people (what flag |= *ptr++ actually does isn't immediately obvious), and some design aspects made me say "oh noes, why u do dis?!". I did a write-up on all of this and saved it, but I wanted to keep this post semi-short.

    From what I can determine, the point of the routine is to iterate over the entire speed[x][y] multi-dimensional array and look for any entries (counters) that are non-zero. If such an entry is found, return 0 (false), else return a non-zero value (true). The speed array is kinda big: speed[720][2] by default. The values are all uint64_t (per the Node struct).

    Hullabaloo aside, this is nothing that nested for-loops and an if() can't handle. You know, like this?

    Code:
    int speed_empty(Node *node) {
      int i, j;
    
      for (i = 0; i < MAX_NSPEED; ++i) {
        for (j = 0; j < MAX_COUNTER; ++j) {
          if (node->speed[i][j]) {
            return 0;
          }
        }
      }
      return 1;
    }
    
    Wow look, no more worries about int sizes, no need for casting, etc.. This exact same methodology is already used in cstats.c in Node_print_speedjs() itself. This just further makes me wonder why speed_empty() was coded the way it was.

    I benchmarked this vs. the referenced speed_empty(). Mine is faster in almost all cases (again: I did a write-up, but saved it to keep the post short), and I tested several compiler flag combinations. cstats is compiled with gcc -O2 (this matters), but with the above code, using -funroll-loops and -finline-functions would be wise (as long as it didn't bloat the binary).

    If you want me to provide an actual patch you can try, just let me know. I make zero guarantees this actually works/behaves properly, I simply wrote standalone C code that mimicked bare-bones cstats.{c,h} and went to town. I may have the return 0/1 values reversed (I doubt it though). But if it does work properly, well, now you have some code that actually is readable.
     
    William Clark likes this.
  38. M_ars

    M_ars Network Guru Member

    It also took me e few hours to follow that code and look up and understand a few things. The code style is very short and shows what is possible (not saying i like that). Especially flag |= *ptr++ and the two-dimensional array caught my attention :)
    Back in school i had one/two friends who loved to code that way for embedded systems/micro processors.
    The only reason i can think of is speed... but i like your way much more with for-for loop to check the array

    Yes, that is what i understand too.

    code looks good. I think i will add some comments maybe and make a testbuild to check if everything is working. As far as i can see, it should work.

    Thank you for your help Koitsu :)
    (It is always better if more people write/check/test code)
    Will provide the patches for Toastman-branches if everything works

    best regards
    M_ars
     
    Jacques likes this.
  39. koitsu

    koitsu Network Guru Member

    @M_ars Yeah, the flag |= *ptr++; code is actually convoluted because of how operator precedence works and what actually gets assigned to flag (the OR operator is only relevant due to how flag is being used). In other words, these all do the exact same thing:

    Code:
    1. flag |= *ptr++;
    
    2. flag |= *ptr;
       ptr++;
    
    3. flag = flag | *ptr++;
    
    4. flag = flag | *ptr;
       ptr++;
    
    Speed-wise, my code is fine. I haven't tested on ARM yet, but at least on x64 with gcc 4.2.1 (yes, 4.2.1), testing 999999999 calls to speed_empty() (original) and speed_empty_koitsu() (mine above):

    Code:
    gcc: speed_empty(): 10.77 sec, speed_empty_koitsu(): 8.50 sec
    gcc -funroll-loops: same as above
    gcc -funroll-loops -finline-functions: same as above
    
    gcc -O2: speed_empty(): 3.89 sec, speed_empty_koitsu(): 4.25 sec
    gcc -O2 -funroll-loops: speed_empty(): 2.56 sec, speed_empty_koitsu(): 2.28 sec
    gcc -O2 -funroll-loops -finline-functions: speed_empty(): 1.12 sec, speed_empty_koitsu(): 1.06 sec
    
    gcc -Os: speed_empty(): 3.89 sec, speed_empty_koitsu(): 3.90 sec
    gcc -Os -funroll-loops: speed_empty(): 3.07 sec, speed_empty_koitsu(): 3.02 sec
    gcc -Os -funroll-loops -finline-functions: same as above
    
     
    Last edited: Feb 9, 2017
    William Clark, M_ars and Jacques like this.
  40. Edrikk

    Edrikk Network Guru Member

    I'm hoping @Toastman you have been able to keep track of all these fixes and can hopefully integrate? The various fixes (and updates) I've seen so far:


    The forum is a sure fire way to lose all this great work!

    [Corrected cstat fix; Changed to M_ars and corrected link]
     
    Last edited: Mar 23, 2017
    Vindicator and Jacques like this.
  41. koitsu

    koitsu Network Guru Member

    @Edrikk -- Correction: that last "cstat fix" should be for M_ars and/or cc (original author circa 2016), and this is the relevant link: http://www.linksysinfo.org/index.php?threads/tomato-toastmans-releases.36106/page-42#post-284795

    My hemming and hawing in the post you linked is over the speed_empty() function, which was written in a way to "flex technical prowess" rather than remain practical (i.e. it works, but there's an easier/cleaner (and faster) way to do it, hence the code I wrote). I don't want to do a patch for anything until M_ars gets back to us on the testing, then I can do some commits for ARM/ARM7/MIPS giving him and cc proper respect for doing the heavy lifting. I could try it out/test on ARM myself, but I've been busy, and I also have a surgery coming up.
     
    Jacques likes this.
  42. Jacques

    Jacques LI Guru Member

    I have just disabled and actived Wireless Client Filter.
    Capture11.PNG
    Alex-iPad a still 1500 km from my home and now no visible that is ok.
    Liliana-iPad is back at home.

    Capture12.PNG
    All IP Phantom have disappeared.
     
  43. M_ars

    M_ars Network Guru Member

    Did flash my main router yesterday with a custom build and everything is working. I just checked it again.
    The little patch/update does hide hosts with no traffic on IP Traffic Last 24 Hours

    files/patches are attached for MIPS & ARM

    Code:
    From: M_ars <M_ars@linksysinfo.org>
    Date: Thu, 9 Feb 2017 18:41:02 +0100
    Subject: [PATCH] cstats: hide hosts with no traffic on IP Traffic Last 24
     Hours - thx to CC and koitsu
    
    see http://www.linksysinfo.org/index.php?threads/tomato-toastmans-releases.36106/page-42#post-284795
    and http://repo.or.cz/tomato-rt-n10.git/commit/ac11ba2312aa7a596a142bf4bcfa53c4deb783df
    ---
     release/src/router/cstats/cstats.c | 18 ++++++++++++++++++
     1 file changed, 18 insertions(+)
    
    diff --git a/release/src/router/cstats/cstats.c b/release/src/router/cstats/cstats.c
    index d386703..70756e8 100644
    --- a/release/src/router/cstats/cstats.c
    +++ b/release/src/router/cstats/cstats.c
    @@ -385,11 +385,29 @@ static void load(int new) {
         }
     }
     
    +int speed_empty(Node *node) {
    +  int i, j;
    +  /*iterate over the entire speed[i][j] two-dimensional array
    +    and look for any entries (counters) that are non-zero.
    +    If such an entry is found, return 0 (false)*/
    +  for (i = 0; i < MAX_NSPEED; ++i) {
    +    for (j = 0; j < MAX_COUNTER; ++j) {
    +      if (node->speed[i][j]) {
    +    return 0;
    +      }
    +    }
    +  }
    +  return 1;
    +}
    +
     void Node_print_speedjs(Node *self, void *t) {
         int j, k, p;
         uint64_t total, tmax;
         uint64_t n;
         char c;
    +
    +    //hide hosts with no traffic on IP Traffic - Last 24 Hours
    +    if (speed_empty(self)) return;
     
         node_print_mode_t *info = (node_print_mode_t *)t;
     

    Attached Files:

  44. Vindicator

    Vindicator Network Guru Member

    @Toastman and @koitsu ( and others who can help :) )

    I need your opinion regarding DNS intercept and DNS over TCP:

    Observation:
    Tomato has an option to intercept DNS requests from LAN (advanced-dhcpdns.asp -> Intercept DNS port).
    This checkbox sets nvram variable nvram.dns_intcpt.
    This variable is then used by /release/src/router/rc/firewall.c to create the following iptable rule (nat table):
    Code:
    726 if (nvram_match("dns_intcpt", "1")) {
    727    ipt_write("-A PREROUTING -p udp -s %s/%s ! -d %s/%s --dport 53 -j DNAT --to-destination %s\n",
    728                         lanaddr, lanmask,
    729                         lanaddr, lanmask,
    730                         lanaddr);
    731    if(strcmp(lan1addr,"")!=0)
    732             ipt_write("-A PREROUTING -p udp -s %s/%s ! -d %s/%s --dport 53 -j DNAT --to-destination %s\n",
    733                         lan1addr, lan1mask,
    734                         lan1addr, lan1mask,
    735                         lan1addr);
    736    if(strcmp(lan2addr,"")!=0)
    737             ipt_write("-A PREROUTING -p udp -s %s/%s ! -d %s/%s --dport 53 -j DNAT --to-destination %s\n",
    738                         lan2addr, lan2mask,
    739                         lan2addr, lan2mask,
    740                         lan2addr);
    741    if(strcmp(lan3addr,"")!=0)
    742             ipt_write("-A PREROUTING -p udp -s %s/%s ! -d %s/%s --dport 53 -j DNAT --to-destination %s\n",
    743                         lan3addr, lan3mask,
    744                         lan3addr, lan3mask,
    745                         lan3addr);
    746 }
    
    This rule will then "catch" all the DNS requests coming from the LAN side of the router and destined to an ip address outside of the LAN subnet, forcing them to be resolved by the router itself. This is useful to intercept DNS requests of LAN clients that are trying to bypass our DNS server.


    Example of the rule applied in iptables (tomato router is at 192.168.1.1):
    Code:
    -A PREROUTING -s 192.168.1.0/255.255.255.0 -d ! 192.168.1.0/255.255.255.0 -p udp -m udp --dport 53 -j DNAT --to-destination 192.168.1.1


    Possible Problem:
    The referred above is only working with DNS requests over UDP. If the DNS UDP answer is truncated, then the DNS query must be done in TCP and these are bypassing the tomato rule.

    The original limit was 512 bytes as defined in the original RFC (and still used by most clients if I'm not mistaken). Larger messages would be truncated and were sent over TCP.

    Now, if both the client and server support the EDNS(0) extension (RFC 6891), the maximum recommended DNS payload over UDP is 4KB.

    Nevertheless, the growing deployment of DNS Security (DNSSEC) and IPv6 has increased response sizes and therefore the use of DNS over TCP (RFC 7766).

    Excerpt from the RFC:


    Example:
    Linux client and tomato router (with DNSSEC enabled):
    note: dig doesn't set an EDNS receive buffer size by default, so it increases the odds of getting truncated messages.

    Code:
    $ dig org. SOA +dnssec
    ;; Truncated, retrying in TCP mode.
    
    ; <<>> DiG 9.9.5-9+deb8u9-Debian <<>> org. SOA +dnssec
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17709
    ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
    
    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags: do; udp: 512
    ;; QUESTION SECTION:
    ;org.               IN   SOA
    
    ;; ANSWER SECTION:
    org.           899   IN   SOA   a0.org.afilias-nst.info. noc.afilias-nst.info. 2012361802 1800 900 604800 86400
    org.           899   IN   RRSIG   SOA 7 1 900 20170303181157 20170210171157 3947 org. HpPtIYMJmk49nfFpKClqsqGnonfjYi9y3GWyM/fhSsqTsY8PkM9Y4jrE
    (...)


    Possible Solution:
    So, my question is: should we add a new line in /release/src/router/rc/firewall.c to also intercept TCP traffic per each lanaddr?

    Code:
    if (nvram_match("dns_intcpt", "1")) {
     ipt_write("-A PREROUTING -p udp -s %s/%s ! -d %s/%s --dport 53 -j DNAT --to-destination %s\n",
                             lanaddr, lanmask,
                             lanaddr, lanmask,
                             lanaddr);
     ipt_write("-A PREROUTING -p tcp -s %s/%s ! -d %s/%s --dport 53 -j DNAT --to-destination %s\n",
                             lanaddr, lanmask,
                             lanaddr, lanmask,
                             lanaddr);
     (...)
    
    Thanks!

    Edit: Forgot to say that the above solution works: I manually inserted the iptables append command in the firewall script tab. The router rules are correctly created and removed and tomato's dnsmasq is capable of receiving/sending DNS over TCP. The above dig example was after applying these changes to the router.

    I only added the commands to append the new rule. The removal is probably done by a flush command when disabling the DNS intercept option. I say probably because I couldn't find where they were being removed in the source code. But for the existing code to also be capable of removing my iptables append command, I suppose its flushing the entire iptables nat table (at least) and recreating it accordingly with the nvram variables.
     
    Last edited: Feb 10, 2017
    M_ars likes this.
  45. RMerlin

    RMerlin Network Guru Member

    Probably, at least that is how I implemented it for my DNSFilter feature.

    https://github.com/RMerl/asuswrt-merlin/blob/master/release/src/router/rc/firewall.c#L5323

    EDIT: dunno if Tomato also implemented it for IPv6, but here's how to do it:

    https://github.com/RMerl/asuswrt-merlin/blob/master/release/src/router/rc/firewall.c#L5356
     
    Vindicator and M_ars like this.
  46. koitsu

    koitsu Network Guru Member

    @Vindicator -- this looks correct (what you propose/the solution). You understand EDNS correctly, and what TCP usage in DNS is for (aside from some other things like AXFR-based zone transfers). One thing I wanted to shed light on:

    The answer you're looking for is iptables-restore. Tomato builds/creates the /etc/iptables file and then runs iptables-restore /etc/iptables > /var/notice/iptables. This is done in router/rc/firewall.c, function start_firewall(). iptables-restore by default flushes all rules/chains before adding rules (to inhibit that, use the -n flag).

    I think RMerlin covered the relevant bits, incl. what to do about IPv6 (which we don't intercept right now, AFAIK, but I'm not sure).
     
    Vindicator likes this.
  47. Vindicator

    Vindicator Network Guru Member

    Thanks @RMerlin and @koitsu !

    I overlooked the IPv6 side of this :(

    Since there is no nat table in IPv6 (l think it was only introduced in kernel 3.9.0 and ip6tables 1.4.18) we can only accept or drop the packet, like what @RMerlin implemented in asuswrt-merlin.

    I was thinking if we could use the TPROXY target for now (mangle table and PREROUTING chain in ip6tables). This could in theory allow us to redirect a packet:

    Code:
    TPROXY
    This target is only valid in the mangle table, in the PREROUTING chain and user-defined chains which are only called from this chain. It redirects the packet to a local socket without changing the packet header in any way. It can also change the mark value which can then be used in advanced routing rules. It takes three options:
    --on-port port
    This specifies a destination port to use. It is a required option, 0 means the new destination port is the same as the original. This is only valid if the rule also specifies -p tcp or -p udp.
    --on-ip address
    This specifies a destination address to use. By default the address is the IP address of the incoming interface. This is only valid if the rule also specifies -p tcp or -p udp.
    --tproxy-mark value[/mask]
    Marks packets with the given value/mask. The fwmark value set here can be used by advanced routing. (Required for transparent proxying to work: otherwise these packets will get forwarded, which is probably not what you want.)
    

    The downside of this is the part:

    For this to work, dnsmasq would need to:

    Code:
          IP_TRANSPARENT (since Linux 2.6.24)
                 Setting this boolean option enables transparent proxying on
                 this socket.  This socket option allows the calling
                 application to bind to a nonlocal IP address and operate both
                 as a client and a server with the foreign address as the local
                 endpoint.  NOTE: this requires that routing be set up in a way
                 that packets going to the foreign address are routed through
                 the TProxy box (i.e., the system hosting the application that
                 employs the IP_TRANSPARENT socket option).  Enabling this
                 socket option requires superuser privileges (the CAP_NET_ADMIN
                 capability).
    
                 TProxy redirection with the iptables TPROXY target also
                 requires that this option be set on the redirected socket.

    So, probably not worth the effort... At least until ip6tables and kernel support for IPv6 nat in tomato\asuswrt in the years to come (fingers crossed :))
     
  48. Sean B.

    Sean B. LI Guru Member

    Nice timing. I just finished patching tproxy to add the target for ipv6, as I already use tproxy for squid.. running a build now.
     
    Vindicator likes this.
  49. Elfew

    Elfew Network Guru Member

    kille72 likes this.
  50. RMerlin

    RMerlin Network Guru Member

    Tomato does not NAT IPv6, it routes it. Hence it must go in the OUTPUT and FORWARD tables.
     
  51. Vindicator

    Vindicator Network Guru Member

    Check. You're confirming my post. So if we would like to intercept the DNS (in IPv6) requests and forward them to the router, just like we do in IPv4 our only option at the moment would be the PREROUTING chain in the MANGLE table, which is supported by the current kernel in tomato and asuswrt, right? Like what @Sean B. is using for squid.

    Thanks
     
  52. RMerlin

    RMerlin Network Guru Member

    No, INPUT and FORWARD. Check the second Github link in my post to see how I do it. The DNSFILTER chain is where I do the actual "redirection".

    Note that I haven't done any extensive test with IPv6, I based most of this portion of DNSFilter on Asus's own code for YandexDNS.
     
  53. Vindicator

    Vindicator Network Guru Member

    @RMerlin don't take me wrong, you clearly know your own code better than anyone, but I looked at your code again and it still appears to me that it either allows or drops the IPV6 DNS requests, I can't find the redirection you mention. Can you help me out?


    Code:
    dnsmode = nvram_get_int("dnsfilter_mode");
    if (dnsmode) {
    /* Allow other queries to the default server, and drop the rest */
    count = get_dns_filter(AF_INET6, dnsmode, server);
    if (count) {
    fprintf(fp, "-A DNSFILTERI -d %s -j ACCEPT\n"
    "-A DNSFILTERF -d %s -j ACCEPT\n",
    server[0], server[0]);
    }
    if (count == 2) {
    fprintf(fp, "-A DNSFILTERI -d %s -j ACCEPT\n"
    "-A DNSFILTERF -d %s -j ACCEPT\n",
    server[1], server[1]);
    }
    fprintf(fp, "-A DNSFILTERI -j %s\n"
    "-A DNSFILTERF -j DROP\n",
    (dnsmode == 11 ? "ACCEPT" : "DROP"));

    For IPv4 you use the DNSFILTER chain which redirects traffic. Check!
    For IPv6 the DNFILTERI (used for INPUT) and DNSFILTERF (for FORWARD) seem to either allow or drop.
    The same appears to be happening for the YANDEXDNS filter.


    This is the last entry I could find in your changelogs:
    (it's old, so I presume somewhere along the way you or Asus changed this)

    Thanks again
     
  54. Sean B.

    Sean B. LI Guru Member

    IPv6 nat ( NAT66 ) does exist in netfilter for kernels 3.something and above. But I can only imagine how interesting that would be to try an backport, if even doable.
     
    Vindicator likes this.
  55. RMerlin

    RMerlin Network Guru Member

    You're correct. I wrote that code over two years ago and never looked at it again since then, so I forgot how it was implemented. The commit is here:

    https://github.com/RMerl/asuswrt-merlin/commit/96d1bcc911a00f7c4aae2d6cbca6af65859db181
     
    Vindicator likes this.
  56. cobrax2

    cobrax2 Reformed Router Member

    guys any way to bind a script to a physical button? i would like to enable/disable the 2.4Ghz when i push the wps button for example on my R7000
    thanks
     
  57. ruggerof

    ruggerof LI Guru Member

    Administration - Buttons/LED. I just don't know if it works with the R7000.
     
    cobrax2 likes this.
  58. cobrax2

    cobrax2 Reformed Router Member

    oh, right, i'm stupid! :)
    thanks
    will try tonight
    any clue as to how can i build a script to do that?
     
  59. ruggerof

    ruggerof LI Guru Member

  60. cobrax2

    cobrax2 Reformed Router Member

    but how to keep track of wether it is on or off? so it will toggle the othe way
    thanks
     
  61. M_ars

    M_ars Network Guru Member

    Hi Vindicator,

    can you make the patch for firewall.c? :)
    A little cosmetic change for the gui (advanced-dhcpdns.asp) is also necessary then: (UDP 53) --> (UDP/TCP 53) or something similar maybe

    For IPv6, drop the packet could be the easy/default way to go I think. If somebody wants something else --> custom script/firewall rules
     
    William Clark likes this.
  62. ruggerof

    ruggerof LI Guru Member

    The command "radio toggle 0" will do what you want.
     
  63. KyleS

    KyleS Addicted to LI Member

  64. Vindicator

    Vindicator Network Guru Member

    @M_ars, @Toastman and @shibby20

    The patch files for IPv4 (MIPS and ARM) are attached to this post.

    These are my first patches, so please bear with me if they aren't OK.

    Code:
    From: Vindicator <>
    Date: Sat, 18 Feb 2017 02:17:41 +0000
    Subject: [PATCH] Current builds only intercept DNS traffic that is sent over
     UDP. This patch adds the interception for TCP.
    
    See the following post for more details:
    
    https://www.linksysinfo.org/index.php?threads/tomato-toastmans-releases.36106/page-42#post-284940
    
    Note: This only applies for IPv4
    
    Signed-off-by: Vindicator <>
    ---
     release/src-rt-6.x.4708/router/rc/firewall.c       | 26 +++++++++++++++++++---
     release/src-rt-6.x.4708/router/rc/restrict.c       |  3 ++-
     .../router/www/advanced-dhcpdns.asp                |  2 +-
     3 files changed, 26 insertions(+), 5 deletions(-)
    
    diff --git a/release/src-rt-6.x.4708/router/rc/firewall.c b/release/src-rt-6.x.4708/router/rc/firewall.c
    index 1d9717b..844a451 100644
    --- a/release/src-rt-6.x.4708/router/rc/firewall.c
    +++ b/release/src-rt-6.x.4708/router/rc/firewall.c
    @@ -761,25 +761,45 @@ static void nat_table(void)
     
             if (wanup) {
                 if (nvram_match("dns_intcpt", "1")) {
    +                // Vindicator 2017: Need to intercept both TCP and UDP DNS requests for all lan interfaces
    +                ipt_write("-A PREROUTING -p tcp -s %s/%s ! -d %s/%s --dport 53 -j DNAT --to-destination %s\n",
    +                    lanaddr, lanmask,
    +                    lanaddr, lanmask,
    +                    lanaddr);
                     ipt_write("-A PREROUTING -p udp -s %s/%s ! -d %s/%s --dport 53 -j DNAT --to-destination %s\n",
                         lanaddr, lanmask,
                         lanaddr, lanmask,
                         lanaddr);
    +                if(strcmp(lan1addr,"")!=0) {
    +                    ipt_write("-A PREROUTING -p tcp -s %s/%s ! -d %s/%s --dport 53 -j DNAT --to-destination %s\n",
    +                        lan1addr, lan1mask,
    +                        lan1addr, lan1mask,
    +                        lan1addr);
     

    Attached Files:

    Edrikk and M_ars like this.
  65. M_ars

    M_ars Network Guru Member

    thx, looks good :)

    i have two little cosmetic improvements/suggestions:
    i think the description is verly large now - very close to the checkbox.
    Code:
    +    { title: 'Intercept DNS port<br>(IPv4 TCP and UDP port 53)', name: 'f_dns_intcpt', type: 'checkbox', value: nvram.dns_intcpt == '1' },
    Can you make it a little shorter? ditch "IPv4" for example?

    Because there is one more place for infos like that, see notes section advanced-dhcpdns.asp
    Code:
    <li><b>Intercept DNS port</b> - Any DNS requests/packets sent out to UDP port 53 are redirected to the internal DNS server.</li>
    Can you update the DHCP notes section and the patch? that would be great :)

    best regards
    M_ars
     
    Last edited: Feb 18, 2017
  66. cobrax2

    cobrax2 Reformed Router Member

    but won't that turn off all radios? i only want to turn on/off one band
    thanks

    edit:
    i used wl -i eth1 radio on and with off
    in the interface it seems to work, but the network doesnt seem to appear to other devices, so it only appears within webinterface
    edit 2:
    also used radio toggle command, it also does something to the web interface but to the actual radio it does nothing
    hmm
    i'll move this to another thread as it pollutes this one, sorry, thought it was an easy task :)
     
    Last edited: Feb 18, 2017
  67. Twincam

    Twincam Serious Server Member

    @cobrax2 No - and I didn't know this either. "radio toggle 0" toggles the 2.4 radio and "radio toggle 1" does the same for the 5.0 radio. Very neat!

    If you want to see it "in action", leave the WebUI on the main "Status" page and use an SSH (PuTTY) session to enter the commands (above) to see the change effected in "real time".

    Edit: On my RT-N66U, the router status lights are also updated to reflect the correct radio (on/off) status.
     
    Last edited: Feb 18, 2017
  68. cobrax2

    cobrax2 Reformed Router Member

    the problem is that it does not do anything on my R7000. i can't see the 2,4 network go down or up on my laptop
     
  69. koitsu

    koitsu Network Guru Member

    The commands and behaviour vary per model of router, so it's no surprise some work for one but not another. Commands you can try, speaking strictly about 2.4GHz (there are 5GHz equivalents but let's just focus on one frequency for now):

    radio toggle 0
    wl -i eth1 radio off
    (re-enable: wl -i eth1 radio on)
    wl -i eth1 down (re-enable: wl -i eth1 up)
    wl -i eth1 out (re-enable: wl -i eth1 up (I hope))

    For the radio command to work, NVRAM variables need to be set in advance. I'm looking at the code right now for ARM and will update this post when I figure out the relevant bit.

    Edit: I only tested radio off 0 (the code for toggle is slightly different but I believe the below advice applies)..

    You actually need two NVRAM variables present for radio off 0 to work:

    1. wl0_radio needs to be a value of 1 (ex. nvram get wl0_radio should output 1; any other output, or not output, will result in radio not functioning)

    2. wl0_ifname needs to be the value of the proper ethX interface that correlates with that radio (ex. nvram get wl0_ifname should output eth1, I think, on this model of router. I'm not sure though -- I do not own an R7000).

    The point here is: radio is essentially a sort of "wrapper" around the wl commands above, but also does things like toggle router LEDs (the function set_radio() does this). It doesn't actually call the wl utility/command, it instead uses Broadcom wl_ioctl() function calls and several other things to interface with the wireless driver directly.
     
    Last edited: Feb 18, 2017
    cobrax2 likes this.
  70. Twincam

    Twincam Serious Server Member

    @CobraX OK. I think that must be because you are using an ARM build (and may be evidence of a bug). I am using MIPS builds - currently, the one indicated in my signature.

    Perhaps other ARM Users can help (or confirm)? Sorry.
     
    Last edited: Feb 18, 2017
    cobrax2 likes this.
  71. cobrax2

    cobrax2 Reformed Router Member

    yes, i've tried exactly those commands. they throw no error, the webui gets updated, but the radio doesnt go offline or online
    thanks
     
  72. koitsu

    koitsu Network Guru Member

    I'm sorry, I cannot reproduce this on Toastman-ARM using an RT-AC56U. wl -i eth1 radio off absolutely turns the radio off -- I can tell because SSID broadcast ceases immediately (verified with several devices), one cannot connect to the SSID, and TX packet counters on the eth1 interface stop incrementing.

    This may be a problem or issue specific to the R7000 model in some way.

    Are you certain eth1 is the 2.4GHz wireless interface on the R7000? Again: I cannot verify because I do not have this router.
     
  73. Mercjoe

    Mercjoe Network Guru Member

    I can tell you that it works fine on my R7000. I went into tools -> Commands and executed it.. Radio was disabled.

    Personally, I use radio off 0 to just turn off the 2.4 ghz radio. Much simpler IMHO
     
  74. ruggerof

    ruggerof LI Guru Member

    Same here: R7000 running v1.28.9008 Toastman-ARM K26ARM USB VPN-64K
     
  75. ambiance

    ambiance Serious Server Member

    At first I thought I was experiencing the same problem, but then I forgot I had recently hooked up another AP and it was just connecting to that one instead. I'm using an R7000 with 1.28.9008 and radio toggle 0, radio off 0, wl -i eth1 radio off, wl -i eth1 down, wl -i eth1 out all worked.
     
    Last edited: Feb 20, 2017
  76. cobrax2

    cobrax2 Reformed Router Member

    damn, then wth mine doesnt work? :(
     
  77. Edrikk

    Edrikk Network Guru Member

    Vindicator likes this.
  78. Vindicator

    Vindicator Network Guru Member

    @Edrikk, another set of patches for @Toastman :)

    These don't replace my previous set, they require them and offer only cosmetic changes to the standard Web GUI (tnx for tip @M_ars), although, I must say, I'm a fan of @Jacky444 work on Advanced Tomato.
     

    Attached Files:

    Edrikk and M_ars like this.
  79. stuffedtiger

    stuffedtiger Reformed Router Member

    Can someone tell me what specifically is fixed by the commit in post #4094? I'm very curious.
     
  80. Vindicator

    Vindicator Network Guru Member

    You need to initialize the header ptr with the new value before using it. Else, you end up writing in the wrong position in nvram (because you would get a wrong magic_offset).

    Code:
    src/linux/universal/linux-4.4/arch/arm/plat-brcm/nvram_linux.c
    r28606     r31160   
    584    584            } else {
    585    585                    offset = nvram_mtd->size - nvram_space;
    586    +                header = (struct nvram_header *)buf;
    586    587                    magic_offset = ((void *)&header->magic - (void *)header);
    587    -                    header = (struct nvram_header *)buf;
    588    588            }
    589    589
    Edit: check post #4096. @koitsu explained it well.
     
    koitsu likes this.
  81. koitsu

    koitsu Network Guru Member

    What is it you want to know? The technical explanation? I provide it in the commit messages in my separate branches, as referenced in post #4096 (i.e. read the github commit messages).

    It affects nvram_commit() (i.e. "nvram commit"), but the actual condition that would trigger the problem I cannot replicate/simulate because of how buf is allocated during real-time using kmalloc() and populated.

    The bottom line is that the result could be magic_offset being very incorrect. This value is later used during NVRAM population. So, based on that, I would say it could end up causing NVRAM corruption (in a very very bad way), if the trigger condition was fully understood/known (see above paragraph). What I can't tell you is "how" this would manifest exactly -- mainly because I don't know how the "NVRAM magic" (not magic as in magicians/magic smoke/mystical things, but the actual magic value itself) plays a role in NVRAM. It may act as a checksum, I simply don't know.

    Use of -Wall should emit a warning that header is being used when uninitialised, but I get the feeling that piece of Broadcom code isn't compiled with -Wall -Werror. It really should be (and that's Broadcom's fault, AFAIT). You can clearly see in the if() block above that header is assigned first, followed by magic_offset, so this is definitely an error/mistake on Broadcom's part.
     
    William Clark likes this.
  82. stuffedtiger

    stuffedtiger Reformed Router Member

    Thank you for the replies. I usually don't click on github links since I'm nowhere near skilled enough to read the code properly. This one's on me. It slipped my mind that there would be comments for the edits.

    It's always a good time reading your posts @koitsu. Thanks again.
     
  83. edusodanos

    edusodanos Serious Server Member

    Does anyone have any idea of when toastman will launch the next compilation including new updates?
     
  84. Twincam

    Twincam Serious Server Member

    @Toastman

    I just upgraded to "tomato-K26USB-NVRAM64K-1.28.0511.5MIPSR2Toastman-RT-N-VPN.trx" on my RT-N66U. All is well with 2x subnets, 2x OpenVPN servers (1xTUN; 1xTAP), "Bandwidth" & "IP Traffic" monitoring. "Scheduler" works for "Reboot" & "Custom" options.

    The issue I reported in #3834 remains (and was not present until "Virtual Wireless" interfaces were defined during configuration. Until they were defined, the "show/hide" status worked correctly for WiFi.). If it were possible to fix this, it would also be nice to provide the same functionality for the "Ethernet Ports State" block. ;)

    My main reason for updating (from v0510.3) was to eliminate the "PPPoE" reconnection "rstats bulge". This appears to be fixed - at least for my low speed (~2.3Mbps down) ADSL connection.

    Thanks.
     
    Last edited: Mar 1, 2017
  85. Nathaniel Cowles

    Nathaniel Cowles Serious Server Member

    Hi Toastman, thanks for your efforts!

    Perhaps anyone can help me. I downloaded these builds fine months ago but now I've tried repeatedly and cannot get a download to start. Thank you for any help.
     
  86. kw_broadens

    kw_broadens Network Newbie Member

    It's not intuitive, but you need to click on the blue FREE DOWNLOAD button to start the countdown timer. I just tried it and it works for me.
    Ken.
     
  87. Nathaniel Cowles

    Nathaniel Cowles Serious Server Member

    I tried that before I posted and the timer wouldn't start but now it works thanks again.
     
  88. Jeffry

    Jeffry Networkin' Nut Member

    I'm not a big fan of 4shared. Toward the end of last year, I got the old "you need to update" some plugin (I think it was flash player which was already up to date) notice after clicking to download the tomato firmware. This happened a few times. Of course, I was farily sure it was fake but I looked at it a bit closer just in case. I can see how someone not too internet savvy might think it was real and as a result, some malware or phishing stuff might get installed.
     
    Last edited: Mar 28, 2017
  89. Nathaniel Cowles

    Nathaniel Cowles Serious Server Member

    Yeah, Flash hurts. Intentionally remove it and you know to avoid those.
     
  90. c4flash

    c4flash New Member Member

     
  91. Monk E. Boy

    Monk E. Boy Network Guru Member

    I love how the malware ads still come up telling me I need to update Flash, even though Flash isn't installed.

    Those ads are why I install ad blockers everywhere. I don't care about ads. It's that the ad networks don't prevent malware being distributed through them. Clean the malware, leaves the ads, ad blockers go away except on extremely limited connections.
     
  92. lepa71

    lepa71 Networkin' Nut Member

    Is there a specific version of netgear stock fw I have to be on before installing initial fw for R7000?

    Is this fw supports hardware acceleration on R7000?

    Thanks
     
  93. c4flash

    c4flash New Member Member

    Presently running Tomato Firmware v1.28.7500 MIPSR2Toastman-VLAN-RT K26 USB Ext on RT-N16.
    Thinking of upgrading to one of the 7511.5 FWs in "RT - Newer MIPSR2 RT-N16 WNR3500L E series etc_1491292865579" on 4shared posted by Toastman a few pages back.

    Any issues I need to know about? Quite happy to stick with 7500 for the time being. (If it ain't broke, don't need to fix it!) I haven't read of any problems with this version since his post.

    Comments, anyone? Stability issues? Glitches?

    Thanks for reading
     
  94. c4flash

    c4flash New Member Member

    Fortunately I don't live in a densely populated neighborhood; still, everyone around me now has n, ac wifi set on "auto". When I first deployed the RT-N16 and enabled wifi mine too was set on auto. As I noticed glitches, I fired up inSSDer and surveyed for a week or so. Since chan 1 was my best option, I just set it for that chan permanently
    and continued to survey. Others would land on chan 1 but they wouldn't stay for long, hopping around from chan to chan. Then new neighbors with an additional roomie moved in next door, both with high power dual band routers, n and ac. So I positioned my router and my antennas for best tx and rx for me, left it on chan 1, 20mhz, kicked up the power a bit, and surveyed. Also checked speedtests during on and off hours (night & day). This worked for me, with the additional placement of the router next to 2 walls blocking most of the 5ghz interference from the 2 new neighbors. But for most bandwidth-intensive operations I just use ethernet cat6 through the RT-N16. Everything that can be ethernet connected, is. Yes, laptops also have wifi but only for light duty. Wifi is for pads and phones.
     
  95. M_ars

    M_ars Network Guru Member

    511.5 and derivates are very good builds in my opinion. Running and running and running... :)

    I dont know any issues so far.
     
  96. c4flash

    c4flash New Member Member


    Thanks for your reply, will be doing a little research to see what features I really want .. and what I can do without, for a lean mean clean RT-N16. Seems comcast will be rolling out docsis 3.1 maybe this year, maybe next. Wonder if I'll need to get a faster router; I'm at 200mbps atm. Couple years ago (?), I installed 7500 above with USB (don't use it), NAS (have my own setup), vlan (wifi uses it, can do without), FTP (I only use it between computers and the NAS, inside the LAN). I agree with Koitsu and Toastman re the purpose of a router is to route. Complexity, merges, bloat, all seem to cause unexpected and often 'mysterious' glitches; at least this is the impression I get from reading these threads.
     
  97. ruggerof

    ruggerof LI Guru Member

    I think that the RT-N16 won't be able to handle this speed.
     
  98. c4flash

    c4flash New Member Member

    I know this is an old post but still valid (if not more) today. I've used https everywhere since it came out. Also noscript (a pita, though). Used to use ABP but replaced it with ublock-origin. Some large sites will fetch dozens of ads and tracking garbage, using YOUR computer's resources to do so. Like USA Today. I think I once counted 128 such monkeys on my computer's back. Javascript is really a nightmare also, but worse, windows explorer can run it on your system. To disable any file.js from running on your pc, just associate .js with notebook. Test it by making a file.js and clicking on it. Any such file will then be pointed to notebook and won't cause any harm. Why windows has a built-in javascript interpreter/enabler to run on pc, outside the browser, is a mystery to me.

    Keep up the good work guys.
     
  99. c4flash

    c4flash New Member Member

    Might be right; fastest was 117mbps with qos/bw limit and extras off; I was the only one online, middle of the night. Fastest direct to modem (sb6120) connection was 171mbps. Which router would you recommend?
     
  100. Joe A

    Joe A Reformed Router Member

Share This Page