Free public redirect project

Discussions about UT99
JackGriffin
Godlike
Posts: 3774
Joined: Fri Jan 14, 2011 1:53 pm
Personal rank: -Retired-

Free public redirect project

Post by JackGriffin »

I got a couple of emails this morning and now have some webcam meetings scheduled tonight to discuss this project with a couple of the bigger names in UT.

I love this damn game. I always have. It got me through my divorce, through moving across the country, through being sick. That simple purchase of Unreal has given me thousands of hours of the best entertainment ever and I don't want that to end yet. A pub redirect project needs to get done, and it's going to be up to us to do it.

I'll research hosting today and find out what we are looking at. What I need from you guys is some commitments to help. Once this thing is up and running I need a rotating staff that understands FTP and redirects. In a nutshell this is how things will work...

There will be a public page where the redirect is located at, exactly like the old free redirects used to be. Everyone is able to use the same link on any servers they like and it will be fully accessible. On the same domain will be a small forum for reporting files, problems, requests, etc. I want this to be as bone simple as possible, maybe even posting without having to create accounts. I'll just need somewhere to post when someone attempts to upload duplicate mismatches, etc.

Potential new redirect files will upload to my home FTP server so they can be checked before being added to the public redirect. This will weed out anything that shouldn't be there and prevent most any 'issues' I've seen from redirects.

What I need from you is either to help judge files submitted and move them to the public repository and/or work the forum as an admin. Files would need to be moved daily (at least) from my home server to the redirect, preferably more often if several people got involved. I expect the small forum to get bombed with spammers but it needs to remain easy to use and post at so it's just going to be something that has to be tolerated and kept clean.

Mirroring UT-Files is just going to have to be someone else's thing, it's too much for me but I can damn sure try to cover the redirect problem if I can garner a little help. Is anyone interested? I would happily train you if you could commit to covering one day every couple of weeks or so.
So long, and thanks for all the fish
User avatar
Feralidragon
Godlike
Posts: 5489
Joined: Wed Feb 27, 2008 6:24 pm
Personal rank: Work In Progress
Location: Liandri

Re: Free public redirect project

Post by Feralidragon »

There's just some input I need to give on this:
- I think that among all the UT available content (maps, mods and whatnot) there are tons and tons of pure garbage.

I think if you're going to undertake this project with other people, perhaps one of the main focus should be separating what's already known and good, and keep those as much as possible, and then focus on the rest with time.
Not only will save a lot of HDD space in the end, and then availability of at least the actual interesting things is assured, as then one can go straight and focus in better stuff and keep around.
I am not saying to run and test every map or mod, but the ones people know and use constantly already (ZP, RX, Siege, MH, known maps + dependencies, etc), those should be the main focus imo, if doable at all ofc.
JackGriffin
Godlike
Posts: 3774
Joined: Fri Jan 14, 2011 1:53 pm
Personal rank: -Retired-

Re: Free public redirect project

Post by JackGriffin »

I couldn't agree more. In fact there needs to be clear limitations on what gets uploaded. I'm not slamming in 5GB at a time from people. They are going to need to message me and say something simple like "I'm going to run an MH server so here are my server files". I'll know then that the 50 DOM maps that are in the mix are not supposed to be there and can be discarded. You can tell pretty quickly by looking at the collection what someone is trying to accomplish. By it's very nature this will be a simplish project. Someone wanting to start an RX clan with three servers and 25GB of redirected files needs to pony up for themselves and get their own. This will be to support the 'little guys' who want to run a server or two.
So long, and thanks for all the fish
UT99.org

Re: Free public redirect project

Post by UT99.org »

billybill wrote:That junk problem haunts me. Forgetting the redirect files for a moment and focussing on all those zipped mods and maps. There must be literally hundreds of double-ups of the same files. Probably not the biggest deal but it does seem like something that eventually should be dealt with.

I would propose a script that unzips everything into their own folders, checks for dependent packages and creates a dependent.txt file in that folder with the names. Rezipping it all up with the readme files,pictures that were there to start with, but all the "universal" packages removed. Or should I save talk like this for the new forum? Big names of UT? Guess I been kept out of the loop lol
User avatar
Feralidragon
Godlike
Posts: 5489
Joined: Wed Feb 27, 2008 6:24 pm
Personal rank: Work In Progress
Location: Liandri

Re: Free public redirect project

Post by Feralidragon »

billybill wrote: I would propose a script that unzips everything into their own folders, checks for dependent packages and creates a dependent.txt file in that folder with the names. Rezipping it all up with the readme files,pictures that were there to start with, but all the "universal" packages removed.
I already made a (very fast) PHP class that detects package dependencies in case anyone wants it, and its purpose was pretty much to extract those and check against a DB to avoid duplicates and mismatches (when I had the idea to build such site in the past), could be useful for new submissions if a site is planned or so.
billybill wrote:Big names of UT? Guess I been kept out of the loop lol
Your name is too short, only 9 letters. Minimal is 10 :tongue:
Higor
Godlike
Posts: 1866
Joined: Sun Mar 04, 2012 6:47 pm

Re: Free public redirect project

Post by Higor »

I agree about keeping the garbage out.

And yes, server admins should be the ones to do the job of double-checking and submitting file lists (for pending approval).
UT99.org

Re: Free public redirect project

Post by UT99.org »

billybill wrote:Yes that script wouldn't take me long to write but if it's already done

And my name, haha. Thats very true, and most of those letters are duplicates :P I actually should write up some sort of history to make up for my lack of releases. Your prerequisites look fine Farali. I mean not to talk trash other members publicly but Jack has some history that should probably be made clear for the good of the project, as well

And I actually derailed the thread a little by talking about the non redirected files. Apologies for that
User avatar
Feralidragon
Godlike
Posts: 5489
Joined: Wed Feb 27, 2008 6:24 pm
Personal rank: Work In Progress
Location: Liandri

Re: Free public redirect project

Post by Feralidragon »

Here it is:
uPackage.zip
(3.17 KiB) Downloaded 175 times
It has only 1 class and uses mainly static methods, and there's also an index.php just to show a basic example of usage (this was mostly meant to be used after the upload of new packages in a shared environment with cpu limits, so it was optimized to be fast and use a very short amount of memory).
Right now it only extracts and validates the basic package stuff: version, signature, name table, export table, import table and dependencies. but it can be extended to do more by connecting the import table with the export table and with the name table of course.
Probably I would leave the code a bit better if I did it nowadays lol, so don't mind some things there.

I tested it with plenty of packages btw (even obfuscated ones), so you shouldn't have issues with at least the vast majority of mods, maps and whatnot there, but in case you do you can send me the package in question to check what's up.

As for "big names", now in a serious tone, I think he meant big admins with already ongoing popular resources perhaps. I am not in the loop either, despite my name having over 10 letters lmao :lol2: , I am just giving input like everybody else.
User avatar
Dr.Flay
Godlike
Posts: 3347
Joined: Thu Aug 04, 2011 9:26 pm
Personal rank: Chaos Evangelist
Location: Kernow, UK
Contact:

Re: Free public redirect project

Post by Dr.Flay »

I've started collecting links and resources to free distribution and mirroring solutions.
I am looking out for services that can be automated and painless for the site-user, and scripted/automated, and free (or a cost-saving) to the admin.
https://vivaldi.net/blogs/entry/file-hosting

:idea: Some of these methods will be viable for cost-saving of redirects, such as the CoralCDN, which caches files for 12 hours
(This can be over-ridden).
Bandwidth is reduced as (by default) the server will only have to serve-up 1 copy of each requested file every 12 hours.
The first person to request a file may see a delay with larger files, but as random people come and go, all requiring different files the system takes the strain.
:thuup: The more popular your server is, the less bandwidth, your redirect will use.
:thudown: Speed will vary, depending on how close you are to the cache sending you the files.
Using the CDN sometimes makes downloads faster for people connecting from half-way round the planet.

Distributed UnrealCDN
It has worked for Aminet for 20 years. No-one needs to host everything, or pay hosting costs.
They host across free University servers all over the world.
The Aminet grew from a PD library, into something bomb-proof.
It is like the Hydras head. Cut 1 off, and another 2 may appear.

Until Universities close or stop running FTP servers, the Aminet lives on.
Every year it looses some, every year it gains some.

Think "Cloud-source" not "single-source".
If 1 person hosts UT-Files again, or a mega-redirect, we will have the same problem again when they can't afford it.
Last edited by Dr.Flay on Fri Mar 07, 2014 7:21 pm, edited 1 time in total.
JackGriffin
Godlike
Posts: 3774
Joined: Fri Jan 14, 2011 1:53 pm
Personal rank: -Retired-

Re: Free public redirect project

Post by JackGriffin »

Flay, we get it. Please let up some.

And yes, Ferali is right. I cashed in some favors to ask admins with resources and abilities to help flesh this idea out. I want to know if it is feasible or not and what would be involved.

I'll tell you all out of the gate though that we can sure cast stones, since that's the way UT is, but at some point someone has to do *something* or nothing gets done.
So long, and thanks for all the fish
User avatar
Feralidragon
Godlike
Posts: 5489
Joined: Wed Feb 27, 2008 6:24 pm
Personal rank: Work In Progress
Location: Liandri

Re: Free public redirect project

Post by Feralidragon »

@DrFlay: While the cloud is good, if it belongs to 1 entity you get back to the exact same problem.
I may own a cloud with lots of people and servers and whatnot, but what if I shut it all down all of a sudden? It doesn't fix that particular problem and we're back to zero.
The solution is to have mirrors in different hosts, either cloud-based or not, it doesn't matter, we just need to have redundancy in *different hosts*, either they're better or worse, it doesn't really matter, what matters first and foremost is to not actually loose the important UT-changing files.
JackGriffin
Godlike
Posts: 3774
Joined: Fri Jan 14, 2011 1:53 pm
Personal rank: -Retired-

Re: Free public redirect project

Post by JackGriffin »

I'm in discussions with BurstNet ( http://www.burst.net/ ) right now to get the redirect set up. Hopefully I'll have it online this weekend. I'll keep you all informed.
So long, and thanks for all the fish
$carface
Skilled
Posts: 212
Joined: Sat Jul 23, 2011 10:58 pm

Re: Free public redirect project

Post by $carface »

Are you looking at taking out a VPS, dedicated or shared hosting? I can get a pretty sweet deal if you let me know the requirements.
JackGriffin
Godlike
Posts: 3774
Joined: Fri Jan 14, 2011 1:53 pm
Personal rank: -Retired-

Re: Free public redirect project

Post by JackGriffin »

A VPS is nice since the cost is lower but I'm worried about the performance. I had leaned towards a weak dedicated server since there would be very little processing power needed, just massive storage and bandwidth requirements. I asked BurstNet to create a package for me following these guidelines since it's a little off the normal request for service.

Ideally it would just be a single folder, accessible to UEngine requests. It would be nice to have a very small forum for reporting files, etc but that can be placed at one of the free forum services or piggybacked off someone already in the UT game. Burst has some nice non-metered services that aren't blazing fast but you won't get dinged for overages and that's what I am exploring with them. Simple is better with this, it needs to be instantly useful for the admins who need it and easy to maintain and update.

If you can set it up through your host then let me know what they offer and I'll compare the prices/packages. I'm heading to work in about 30 minutes so I can't read these forums until tonight (Gov't firewall, it's one thing they can do right lol). Email me if you have something you want me to look at as I can access my hotmail.
So long, and thanks for all the fish
$carface
Skilled
Posts: 212
Joined: Sat Jul 23, 2011 10:58 pm

Re: Free public redirect project

Post by $carface »

I pmd you with some possiblities.

If you are taking a VPS or even dedi, don't rush it. I can definitely get you something at a good price...
Post Reply