Enable External Storage Connection for XenForo 2 Attachments

Enable External Storage Connection for XenForo 2 Attachments

  • Thread starter Thread starter xenvn
  • Start date Start date
Enable-External-Storage-Connection-for-XenForo-2-Attachments.webp


The External Storage Connection feature allows you to store XenForo 2 attachments on an external PHP hosting server automatically (the PHP server only needs to support basic functions: curl_version, openssl_encrypt, openssl_decrypt, md5_file, fread, fopen. No SQL required).

Benefits of this:
Reduce the load on your main server, you can buy cheap hosting or build your own server at home to act as a secondary server to store your attachments.

Supports 2 types of connections:
* URL to External Storage Server:

- The path to the storage.php file on your external storage server, which can be a domain name or an ip address. For example: https://your-storage.com/folder/storage.php
- Download the storage.php file, open it and set: CONNECTION_KEY_CODE, STORAGE_FOLDER, STORAGE_FOLDER_IN_PARENT_FOLDER. Then upload it to your external storage server at any location (the server must support PHP). To use Apache X-Sendfile, you need to setup: APACHE_X_SENDFILE and require the server to support.
* Path to External Storage Folder:
- Enter the local path to the data storage directory (for example, the mount path to the directory in the secondary drive) and it must be in the same directory where Xenforo is installed. For example: storage
- Set Set EXTERNAL_FOLDER same as STORAGE_FOLDER in cache.php file (if you use Fast Display Images)

Automatic and Manual Processing:
  • Files can be automatically processed every 15 minutes by cron job (External Storage Cron Job).
  • You can run it manually via: Tools -> Rebuild caches -> [XenVn] Rebuild Data -> Move attachments to External Storage / Restore attachments to Internal Storage (It may take a long time if there are many files)
  • Or move each file you want at: Contents -> Attachments.
Attachments will be processed in ascending order of attachment_id. Newly updated attachtents will be stored on your main server.
You can see the list of attachments on External Storage at: Contents -> Attachments -> Enable Filter Storage

Forum, Resource, Direct Message and Profile Post Support
Apply external storage to specific forums, resource categories, direct messages and profile post.

Flexible Filtering:
  • Filter attachments by file extensions.
  • Apply minimum and maximum file size thresholds.
PHP Storage Server Compatibility
Requires a simple storage.php endpoint on your external PHP server to handle the connection. The PHP server only needs to support basic functions: curl_version, openssl_encrypt, openssl_decrypt, md5_file, fread, fopen. No SQL connection required.

Chunked Attachment Range support
This feature allows attachments to be downloaded in chunks using HTTP Range requests, letting users resume interrupted downloads and partially download large files without starting over.
The system supports three delivery methods: X-Sendfile (Apache) and PHP Chunked Streaming. Use of X-Sendfile requires that the web server is configured with these modules enabled.

To set up X-Sendfile (Apache) on the server, please refer to the article:
https://xenvn.com/threads/attachment-download-acceleration-for-xenforo.5660/
https://xenvn.com/threads/fast-display-images-for-xenforo.5831/

Secure, Safe and Reliable:
  • Synchronize attachment folder structure between Xenforo server and External Storage server.
  • The system checks file integrity when moving between 2 servers.
  • Easily Move/Restore Attachments by Automatic or Manual.
Note:
When an attachment is stored on External Storage, it will be ignored by the Rebuild Cache tools related to the file such as: Optimize attachments, Rebuild attachment thumbnails,.. and [XenVn] Rebuild Data tools.
The External Storage feature can store both images and display them directly on your forum normally. But do not abuse it when not necessary.

Activation and Usage Instructions:
1.
Go to XenVn Setting -> Storage Tab
2. Enable the option: Enable External Storage Connection

3.
If you select External Storage Type as: URL to External Storage Server
3.1.
Download the file "storage.php" in the description or by the following link:
admin.php?xv/external-storage&download=1
3.2. Open the file and set 4 parameters: CONNECTION_KEY_CODE, FOLDER
+ CONNECTION_KEY_CODE: Secret code to establish a connection between 2 servers, you choose.
+ STORAGE_FOLDER: Folder to store attachments (not public), and it must be in the same directory where Xenforo is installed.
+ STORAGE_FOLDER_IN_PARENT_FOLDER: By default, it's false. The STORAGE_FOLDER directory is located at the same level as where you save your storage.php file. If it's true, the STORAGE_FOLDER setting is in the parent directory.
+ APACHE_X_SENDFILE : Enable the Apache X-Sendfile feature. Server support required. Set false to use PHP Chunked Streaming (default).
3.3. Connection Key Code: Set the same security code you set for CONNECTION_KEY_CODE
3.4. External Storage URL: The path to the storage.php file on your external storage server, which can be a domain name or an ip address. For example: https://your-storage.com/folder/storage.php
3.5. Shared Folder Same Serve: If the External Storage URL shares your main server, set this value to also be the path to STORAGE_FOLDER to speed up internal file processing. Use only for the URL to External Storage Server option. (This feature is useful when your server has two network connections. Large files will be served to users via the secondary network connection using an External Storage URL).

4. If you select External Storage Type as: Path to External Storage Folder
4.1 External Storage URL:
Enter the local path to the data storage directory (for example, the mount path to the directory in the secondary drive), and it must be in the same directory where Xenforo is installed. For example: storage
4.2 No need to use storage.php, Set EXTERNAL_FOLDER same as STORAGE_FOLDER in cache.php file (if you use Fast Display Images)

5. External Storage Cron Job: Enable automatic Move/Restore of attachments every 15 minutes via Cron Job.
6. Optional settings: Apply Forum / Resource, Apply Direct Message, Apply Profile Post, Apply Attachment Extensions, Apply Minimum Attachment Size, Apply.
7. Once the setup is complete, click the Save button.
8. Click the "click here to check the connection" link at the top to check and establish a connection to the external data storage server.
Or this link: admin.php?xv/external-storage
9. If the message "Setup complete" appears, everything is ready and you can use it.
10. Otherwise, if error messages appear, you need to follow the message instructions to complete the setup process.

Stop Using External Storage Connection:
To stop using the external storage server and move all files back to your Xenforo server, follow these steps:
1. In the External Storage Cron Job section, select Disable or Restore attachments to Internal Storage.
2. Go to Tools -> Rebuild caches -> [XenVn] Rebuild Data -> Select: Restore attachments to Internal Storage -> Click Run and wait.
3. The files from the external storage server will be moved to your Xenforo server.
4. The moving process may take a long time, depending on the number of files on the external storage server.
5. Finally, turn off the Enable External Storage Connection function.
If you don't need to move quickly, set up Cron Restore attachments to Internal Storage, and wait a few days, the files will be gradually moved back to your main server.

Enable External Storage Connection for XenForo 2 Attachments-1.webp



Enable-External-Storage-Connection-for-XenForo-2-Attachments-2.webp Enable-External-Storage-Connection-for-XenForo-2-Attachments-3.webp
 
Last edited:
I'm aware that using PHP for authentication and file transfer can lead to high server loads. Could you consider adding an option for Nginx X-Accel-Redirect or Apache X-Sendfile when they're available? PHP would still handle authentication with the Nginx backend, and then Nginx would return the signed link.

I've been waiting for Xenforo to support pre-signed URLs for S3 storage, but they haven't implemented it.
 
Back
Top Bottom