TurnKey server behind Nginx reverse proxy doesn't work with iOS/Safari

[JedMeister]

Following some discussion and research, it appears that there is an issue (in Nginx? Safari/iOS? Apache?) when an Apache server is behind an Nginx reverse proxy.

The workaround is to edit the nginx config (for the reverse proxy). First find the location block - it should look something like this:

location / {
    proxy_pass https://BACKEND_SERVER_IP:443;
    [...]
}

Then within that (i.e. after the first line and before the closing }):

    proxy_hide_header Upgrade;

[debemdeboas]

I'd like to add that this is affecting the TurnKey Samba/NFS OS too, where iOS devices cannot connect to Samba shares whereas Windows/Android devices can.

[JedMeister]

Thanks for reporting.

I'd like to add that this is affecting the TurnKey Samba/NFS OS too, where iOS devices cannot connect to Samba shares whereas Windows/Android devices can.

I would expect the http/https parts (not Samba/NFS etc directly) to have the same issue. The issue occurs when an Apple device connects to an Apache server via Nginx reverse proxy. And we use Apache to provide the landing page and host WebDAV-CGI.

As about ~70-80 appliances are in that category, I won't add them all, but I did add fileserver.

Having said that, you explicitly note "Samba share". Do you mean via SMB? Or via WebDAV? If WebDAV, then as per above, it's still Apache doing the serving (WebDAV connects to the Samba share locally).

As a bit of an aside, I wasn't even aware that reverse proxying Samba would work!? A quick google confirms that indeed it should be possible. Although it's probably still not a good idea, as SMB in general and Samba specifically are not designed to be used via public unfiltered internet

If you were explictly referring to Samba (via SMB) then I'm curious if the same workaround works? It should work for WebDAV as the underlying issue should be the exact same.

[debemdeboas]

Ah, I forgot to mention that, as far as I know, I’m not behind any reverse proxy. My setup is a ProxMox server + TurnKey Fileserver LXC.

…

________________________________ From: Jeremy Davis ***@***.***> Sent: Wednesday, November 22, 2023 12:49:28 AM To: turnkeylinux/tracker ***@***.***> Cc: rafael de bem ***@***.***>; Comment ***@***.***> Subject: Re: [turnkeylinux/tracker] TurnKey server behind Nginx reverse proxy doesn't work with iOS/Safari (Issue #1861) Thanks for reporting. I'd like to add that this is affecting the TurnKey Samba/NFS OS too, where iOS devices cannot connect to Samba shares whereas Windows/Android devices can. I would expect the http/https parts (not Samba/NFS etc directly) to have the same issue. The issue occurs when an Apple device connects to an Apache server via Nginx reverse proxy. And we use Apache to provide the landing page and host WebDAV-CGI. As about ~70-80 appliances are in that category, I won't add them all, but I did add fileserver. Having said that, you explicitly note "Samba share". Do you mean via SMB? Or via WebDAV? If WebDAV, then as per above, it's still Apache doing the serving (WebDAV connects to the Samba share locally). As a bit of an aside, I wasn't even aware that reverse proxying Samba would work!? A quick google confirms that indeed it should be possible. Although it's probably still not a good idea, as SMB in general and Samba specifically are not designed to be used via public unfiltered internet If you were explictly referring to Samba (via SMB) then I'm curious if the same workaround works? It should work for WebDAV as the underlying issue should be the exact same. — Reply to this email directly, view it on GitHub<#1861 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AILGQ2NYF6DY67UZRRQDE3TYFVY4RAVCNFSM6AAAAAA6LNTV32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRSGA2DKMBVGA>. You are receiving this because you commented.Message ID: ***@***.***>

[debemdeboas]

Yet again, I forgot to answer all of your questions; I read your reply as soon as I woke up, so my mind wasn’t 100%. My Samba shares (via smb://<address> on the iPhone Files app) don’t connect at all. I’ve tried a plethora of different configurations (e.g. setting vfs fruit aces to no - or something similar, I don’t recall specifics), and none of them work. I’ve noticed two scenarios: * When connecting with smb://<address>, I encounter a “Socket is not connected” error * When using smb://<address>/<share path>, I can connect but I can’t access the files. I can only see the root share path, and it’s set as “read only” (it shouldn’t, this share is writeable by other devices) After what you said about the Web interface, I decided to try and access the files via the WebDAV CGI file manager and, as you’ve already stated, nothing pops up - no files, nothing. Funny thing is I can list the files when using the Webmin interface under Tools -> File manager; however, the Webmin interface doesn’t like being accessed on mobile it seems and the left menu doesn’t “stick”, it appears and disappears, so moving around was a pain. If you like, I can share my Samba configuration file, and I’m open to helping in any way that I can. Some background: I used to have a TrueNAS Scale instance but migrated to Proxmox. On TrueNAS, Samba worked as well as it can (not very), so I ruled out any device-specific problems. Sadly, I don’t have access to that Samba configuration file anymore as I’ve wiped those drives.

…

________________________________ From: Rafael Almeida de Bem ***@***.***> Sent: Wednesday, November 22, 2023 7:05:02 AM To: turnkeylinux/tracker ***@***.***>; turnkeylinux/tracker ***@***.***> Cc: Comment ***@***.***> Subject: Re: [turnkeylinux/tracker] TurnKey server behind Nginx reverse proxy doesn't work with iOS/Safari (Issue #1861) Ah, I forgot to mention that, as far as I know, I’m not behind any reverse proxy. My setup is a ProxMox server + TurnKey Fileserver LXC.

________________________________ From: Jeremy Davis ***@***.***> Sent: Wednesday, November 22, 2023 12:49:28 AM To: turnkeylinux/tracker ***@***.***> Cc: rafael de bem ***@***.***>; Comment ***@***.***> Subject: Re: [turnkeylinux/tracker] TurnKey server behind Nginx reverse proxy doesn't work with iOS/Safari (Issue #1861) Thanks for reporting. I'd like to add that this is affecting the TurnKey Samba/NFS OS too, where iOS devices cannot connect to Samba shares whereas Windows/Android devices can. I would expect the http/https parts (not Samba/NFS etc directly) to have the same issue. The issue occurs when an Apple device connects to an Apache server via Nginx reverse proxy. And we use Apache to provide the landing page and host WebDAV-CGI. As about ~70-80 appliances are in that category, I won't add them all, but I did add fileserver. Having said that, you explicitly note "Samba share". Do you mean via SMB? Or via WebDAV? If WebDAV, then as per above, it's still Apache doing the serving (WebDAV connects to the Samba share locally). As a bit of an aside, I wasn't even aware that reverse proxying Samba would work!? A quick google confirms that indeed it should be possible. Although it's probably still not a good idea, as SMB in general and Samba specifically are not designed to be used via public unfiltered internet If you were explictly referring to Samba (via SMB) then I'm curious if the same workaround works? It should work for WebDAV as the underlying issue should be the exact same. — Reply to this email directly, view it on GitHub<#1861 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AILGQ2NYF6DY67UZRRQDE3TYFVY4RAVCNFSM6AAAAAA6LNTV32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRSGA2DKMBVGA>. You are receiving this because you commented.Message ID: ***@***.***>

[JedMeister]

I read your reply as soon as I woke up, so my mind wasn’t 100%

[...]

No problem 😁 - it happens to the best of us...

Thanks for the extra info.

Whilst the behaviour you're reporting appears quite similar on face value, it seems to me that the issue you're hitting with SMB is a different bug. In contrast to this issue (Apache behind Nginx proxy via Apple web browser) the SMB connection bug you're hitting seems to explicitly be on Apple's side (it's unclear who's responsibility this Apache/Nginx/Apple issue "belongs" to).

As such, I've opened a new issue, reposted your info and will respond more over there: #1882

[TimeJunkie01]

JedMeister,
I have hit this problem using turnkey-wordpress-18.0-bookworm-amd64 behind NPM v2.11.2.
All iPhone browsers fail (Chrome, Safari, Firefox). I added the "proxy_hide_header Upgrade;" directive in NPM and then I am able to connect to the site, however all media does not display (only text). The site works fine from Windows/Mac/Linux systems. Also 10+ other Wordpress sites work that are not turnkey based.

[JedMeister]

@TimeJunkie01 apologies for (ridiculously) slow response.

So Apache is being reverse proxied by NPM? I assume so as this issue is explicitly regarding TurnKey behind an Nginx reverse proxy. Or perhaps I misunderstand?

Whilst that is not completely unreasonable to have NPM internet facing, personally I'd be doing it the other way around. I.e. put NPM behind Apache (or some other specific web server - configured to reverse proxy NPM). My main reason for that is that Apache (and/or any other mature web server) is first and foremost designed to be public internet facing. So it's "one job" is to deal with the potential problems and security risks that come with running in a hostile environment.

Regardless, it's your server and you have every right to configure it as you want.

So assuming NPM is your frontend, then given this discussion, my guess is that it's something to do with the communication between NPM and Apache.

It would probably be useful to see what is happening to the assests (i.e. pictures in your case). Try enabling "web dev tools" in one of the problem browsers on iOS and look at the "network traffic tab" and reload the page. I'd expect the images to be giving a 404 (not found).

If that's the case, then check both the NPM and Apache logs. I'd expect some sort of HTTP error in NPM logs - perhaps a 5xx? If NPM logs aren't showing anything that looks obvious, then I'd image that there are NPM options to make logging more verbose (sorry I'm not super familiar with NPM).

As per your note re other similar server config, using non-TurnKey backend, assuming that it's a very similar set up - i.e. WP running on Apache behind NPM - then a likely cause is non default Apache config we ship with. In particular I'd suspect the security hardening might be the cause? As a backend server none of that should be relevant, so try disabling the relevant mods and config that provide (most if not all of) the additional security measures:

a2disconf security
a2dismod evasive
a2dismod security2
systemctl restart apache2

Another thing that might be useful is comparing Apache config - both from the TurnKey server and the other "working" server:

apachectl -S # virtual host info overview
apachectl -M # enabled modules

You can actually dump the full Apache config like this:

a2enmod info
systemctl stop apache2
apache2ctl -DDUMP_CONFIG

mod_info docs/directives might be worth looking at?: https://httpd.apache.org/docs/2.4/mod/mod_info.html

Hopefully something there helps. If you've already solved it, please post back with any info you have. If you've just moved on, then hopefully this is of use to someone else. 😁

[cycu85]

adding
proxy_hide_header Upgrade;
to proxy causes the page to load but without images

the solution is to add the line
Header unset Upgrade
to /etc/apache2/apache2.conf