Subeta should get a new cluster...

[Haywire]

I think subeta needs new servers, and seeing as how keith has billions of munnies from Subeta, he should get something like this. Note, this setup could power up to 10 marapets.

1) Nameservers & Filters (DDoS/IP Ban)
- Intel Core 2 Extreme QX9650 Quad Core Processor
- 6GB DDR2 DualChannel Memory

2) Load Balancer (Hardware)

3) NFS/DHCP1
- Intel Core 2 Extreme QX9650 Quad Core Processor
- 18-20GB DDR2 Dual Channel Memory
- 1TB SATA2 HD

4) NFS/DHCP2
- Intel Core 2 Extreme QX9650 Quad Core Processor
- 18-20GB DDR2 Dual Channel Memory
- 1TB SATA2 HD

5) MySQL 1
- Intel Core 2 Extreme QX9650 Quad Core Processor
- 8GB DDR2 Dual Channel Memory
- 80GB SATA2 HD

6) MySQL 2
- Intel Core 2 Extreme QX9650 Quad Core Processor
- 8GB DDR2 Dual Channel Memory
- 80GB SATA2 HD

7) WWW0
- Intel® Core™2 Duo Processor E6400
- 6GB DDR2 Dual Channel Memory
- 80GB SATA2 HD

8) WWW1
- Intel® Core™2 Duo Processor E6400
- 6GB DDR2 Dual Channel Memory
- 80GB SATA2 HD

9) WWW2
- Intel® Core™2 Duo Processor E6400
- 6GB DDR2 Dual Channel Memory
- 80GB SATA2 HD

10) WWW3
- Intel® Core™2 Duo Processor E6400
- 6GB DDR2 Dual Channel Memory
- 80GB SATA2 HD

11) MySQL Cluster Node
- Intel Core 2 Extreme QX9650 Quad Core Processor
- 18-20GB DDR2 Dual Channel Memory
- 80GB SATA2 HD

12) MySQL Storage
- Intel Core 2 Extreme QX9650 Quad Core Processor
- 18-20GB DDR2 Dual Channel Memory
- 2* 1TB SATA2 HD Raid

----------------------------------------------

Dedicated OC12 Fiber Optic Connection
UPS Capable of 4 Hours

Est. Cost for Servers:
126GB DDR2 Dual Channel (4GB Sticks): $22,522.5 (Est. Kingston KVR667D2D4F5K2/8G)
4* 1TB SATA2 HardDrives: $1,266.76 (Est. Saegate Barracuda 7200.11 32MB Cache)
7* 80GB SATA2 HardDrives: $381.64 (Ext. Saegate Barracuda 7200 2MB)
7* Intel Core 2 Extreme QX9650 Quad Core Processor: $7,874.93 (Est.)
4* Intel® Core™2 Duo Processor E6400: $998.64

Est. Total: $33,344.47
(Not including Motherboards, RackMounts & Load Balancer)
(Not including UPS, Internet Connection & Power)

----------------------------------------------

Clustered the above setup is capable of powering a large-scale website, with close to 6.4Million Active Connections. If used for web-hosting, capable
of over 500 websites. Both uses will show next to no lag, if configured correctly. Performance exceeding quotation is possible.

http://www.celossupreme.com/hmm.JPG

[VPSfanatic]

Subeta is the World's biggest Kronik petsite.

[Haywire]

Kronik petsite? Noooo. Subeta uses CreatureWorld scripts, and it is a proven fact.

[VPSfanatic]

So they stole them. Even better :)

[Haywire]

Magik and another CW Member were running a fansite on the same server as CW, and decided to hack it for it's scripts. They got them, and Keith decided to use them. Long story short, Subeta was born from CW Scripts.

[VPSfanatic]

Well Keith is really lazy. Not surprising he stole them.

[Ender]

Thanks for the topic starter, obviously people like us love to talk about these kinds of things. I think your post is awesome! Best iv seen in awhile in terms of designing a new network at what i assume your experience level is! Hope you respond back and we get to talk in the future!

Couple problems

1) too powerful for nameservers, not specilized enough for routers, and you need redundancy at your routing level first. Im saying your routing because you can not be ddos filtering / IPBanning otherwise.

2) Sure, what kinda load balancer? Per connection? Per http request? They are nice either way, no real problems here besides you need 2 of them again if they are going to sit in your route stream aka if they die your site dies. Not much problems here

3) and 4) First problem not enough storage space, subeta uses something along the lines of 3-4 TB of storage right now. (i cant give exact numbers) Yes ok you can do raid instead on each machine, you can get more harddrives. Ok but now redundancy again, what happens when NFS1 or NFS0 goes down? We need a clustered storage solution witch we have built, but this is the foces of our current problems. As you can google normal clustered storage solutions are 100k +

5) and 6) Ok nice, not enough room for mysql log files, maybe you were thinking about putting them on NFS? (witch i currently do, and everyone says is a bad idea but see our storage requirements above). Also from your mention of a cluster later ndb, i assume you would round robin the mysql requests between the 2 mysql servers (the entire point of an ndb table) Would you do this with mysql-proxy? or rewrite the application to do the round robin? Both not quick fixes.

7-10) Sure nice.

11) Mysql cluster you need 2 datanodes for redundancy (another strong point of using ndb). Another problem here is way too much processing power. The ndbd deamon is pretty lightweight on cpu usage, its also not very multithreaded. It basicly uses up a shitload of memory and idles on the cpu usage and just serves data over the network.


12) what is this for? Maybe it should be a 2nd mysql cluster node as stated above.

Btw this is the closest thing iv ever seen someone produce in coming up with an real enterprise solution. I can see you put a lot of work into it. Il update Subeta servers - Wikibeta (http://wikibeta.mbif.net/index.php/Subeta_servers) for you.


PS: small edit i forgot. Of course its 90% about your application your working with, and how many places its doing something it shouldnt be, (like a million) :) You can do a billion active connections with 1 of thouse servers, if it all was is a tiny static file. 1 bad join in your application can bring any server to its knees even if its a 32 cpu top of the line 100G of memory box:
::Tyan Computer Corporation:: Server Barebones detail
(http://tyan.com/product_barebones_detail.aspx?pid=338) hehehe

[Keith]

Kronik petsite? Noooo. Subeta uses CreatureWorld scripts, and it is a proven fact.

Proven.. where? o_o