Showing posts with label scale. Show all posts
Showing posts with label scale. Show all posts

Friday, March 9, 2012

How large does MSSQL scale?

I was at a bbq party last night, and one my close friends, a senior lead
developer with tons of MS certs from MCSD to MCSE and author of a Microsoft
book, told all of us that, "Microsoft SQL Server **does not** scale beyond 2
servers pointing to the same database".
This was a complete shocker to me. Is he correct?
We're building an app that needs to scale up as the demand increases. It
will probably be set up regionally with several SQL servers on the US East
and West coasts and China, and they all need to use the same database.
Is my friend right in saying that MSSQL cannot have more than 2 servers
pointing to the same database? He said Oracle on Linux would be a better fit
for our project.
Or is there some way to have these regional servers share the same database?
SQL Server 2000 installed on a Windows Server 2003 cluster can run on 4
nodes. Though, a database will only run on one server at a time (technically
its the instance that run only on one machine at a time).
See http://www.microsoft.com/usa/webcasts/ondemand/1751.asp for more
information
So 4 servers (8 servers with 64 bit support) can point to the same database,
but only 1 at point can use it. Hopefully that makes sense.
Cheers,
Rod
MVP - Windows Server - Clustering
http://www.nw-america.com - Clustering
"Scott" <nospam.scott@.scottkeen.com> wrote in message
news:uvhxcv%23dEHA.3916@.TK2MSFTNGP11.phx.gbl...
> I was at a bbq party last night, and one my close friends, a senior lead
> developer with tons of MS certs from MCSD to MCSE and author of a
Microsoft
> book, told all of us that, "Microsoft SQL Server **does not** scale beyond
2
> servers pointing to the same database".
> This was a complete shocker to me. Is he correct?
> We're building an app that needs to scale up as the demand increases. It
> will probably be set up regionally with several SQL servers on the US East
> and West coasts and China, and they all need to use the same database.
> Is my friend right in saying that MSSQL cannot have more than 2 servers
> pointing to the same database? He said Oracle on Linux would be a better
fit
> for our project.
> Or is there some way to have these regional servers share the same
database?
>
|||I am not sure exactly what your asking by this statement: "several SQL
servers on the US East
and West coasts and China, and they all need to use the same database".
SQL Server is a relational database server application. It is pretty much
an all inclusive app meaning that the database(s) are all owned by one SQL
server. You don't have SQL Server running on a server in Atlanta that owns
a database running on another Server in Chicago. With the exception of
using Distributed Partitioned Views a single SQL Server hosts one or more
databases on the same server as itself. Now don't take that to mean you can
not have other Servers (including SQL Server) connected to it. There are
plenty of SQL applications out there that have literally thousands of users
all over the world. I have personally worked on SQL Servers that handles
thousands of transactions per second, many Terabytes of data and thousands
of users. So you need to be a little clearer on exactly what you mean by
not scaling.
Andrew J. Kelly SQL MVP
"Scott" <nospam.scott@.scottkeen.com> wrote in message
news:uvhxcv%23dEHA.3916@.TK2MSFTNGP11.phx.gbl...
> I was at a bbq party last night, and one my close friends, a senior lead
> developer with tons of MS certs from MCSD to MCSE and author of a
Microsoft
> book, told all of us that, "Microsoft SQL Server **does not** scale beyond
2
> servers pointing to the same database".
> This was a complete shocker to me. Is he correct?
> We're building an app that needs to scale up as the demand increases. It
> will probably be set up regionally with several SQL servers on the US East
> and West coasts and China, and they all need to use the same database.
> Is my friend right in saying that MSSQL cannot have more than 2 servers
> pointing to the same database? He said Oracle on Linux would be a better
fit
> for our project.
> Or is there some way to have these regional servers share the same
database?
>
|||Thanks for the replies.
The concern was raised by the Linux guy in our dev group. I'm a developer,
not a DBA, however I'm responsible for making recommendations on what
technology we need to use for the app.
The Linux guy says, "Microsoft products don't scale well as demand
increases". Broad-sweeping comments like that are very dangerous for the
project, and I need to have my facts right when I make my technology
recommendations to management.
The concern is with what I've been told will be about tens of thousands of
users around the globe (mostly from China and the US coasts) hitting the
database at the same time with complex updates and selects. The concern is
that if SQL Server starts to bog down, then we should be able to throw on
more hardware to spread the load of the requests coming in and going out.
What can we do, so that as the demand on the database increases we'll be
able to throw hardware at the problem to distrubute the load?
Thanks,
Scott
"Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
news:uImmR5$dEHA.1652@.TK2MSFTNGP09.phx.gbl...
> I am not sure exactly what your asking by this statement: "several SQL
> servers on the US East
> and West coasts and China, and they all need to use the same database".
> SQL Server is a relational database server application. It is pretty much
> an all inclusive app meaning that the database(s) are all owned by one SQL
> server. You don't have SQL Server running on a server in Atlanta that
owns
> a database running on another Server in Chicago. With the exception of
> using Distributed Partitioned Views a single SQL Server hosts one or more
> databases on the same server as itself. Now don't take that to mean you
can
> not have other Servers (including SQL Server) connected to it. There are
> plenty of SQL applications out there that have literally thousands of
users[vbcol=seagreen]
> all over the world. I have personally worked on SQL Servers that handles
> thousands of transactions per second, many Terabytes of data and thousands
> of users. So you need to be a little clearer on exactly what you mean by
> not scaling.
> --
> Andrew J. Kelly SQL MVP
>
> "Scott" <nospam.scott@.scottkeen.com> wrote in message
> news:uvhxcv%23dEHA.3916@.TK2MSFTNGP11.phx.gbl...
> Microsoft
beyond[vbcol=seagreen]
> 2
East
> fit
> database?
>
|||SQL Server scales much better up than out. By that I mean you don't add
more servers but more processors, memory etc. If you really think you will
be in a situation where you will have to support thousands of users you
should be looking at hardware in the Unisys ES7000, HP Superdome range. You
can even get them in 64 Bit and they will scale up to 32 and 64 processors
with as much ram as you need. There are already a lot of people at or
beyond that level with SQL Server and with the proper hardware and PLANNING
it should not be a problem. If you have people who lack vision and have
never done something of this scale you may want to seek outside advise to
ensure it is a success. If you bring in a 4 proc box now and tomorrow it
can't handle the load you might indeed have to replace the hardware
altogether...
Andrew J. Kelly SQL MVP
"Scott" <nospam.scott@.scottkeen.com> wrote in message
news:uQeqo9FeEHA.2384@.TK2MSFTNGP09.phx.gbl...[vbcol=seagreen]
> Thanks for the replies.
> The concern was raised by the Linux guy in our dev group. I'm a developer,
> not a DBA, however I'm responsible for making recommendations on what
> technology we need to use for the app.
> The Linux guy says, "Microsoft products don't scale well as demand
> increases". Broad-sweeping comments like that are very dangerous for the
> project, and I need to have my facts right when I make my technology
> recommendations to management.
> The concern is with what I've been told will be about tens of thousands of
> users around the globe (mostly from China and the US coasts) hitting the
> database at the same time with complex updates and selects. The concern is
> that if SQL Server starts to bog down, then we should be able to throw on
> more hardware to spread the load of the requests coming in and going out.
> What can we do, so that as the demand on the database increases we'll be
> able to throw hardware at the problem to distrubute the load?
> Thanks,
> Scott
>
> "Andrew J. Kelly" <sqlmvpnooospam@.shadhawk.com> wrote in message
> news:uImmR5$dEHA.1652@.TK2MSFTNGP09.phx.gbl...
much[vbcol=seagreen]
SQL[vbcol=seagreen]
> owns
more[vbcol=seagreen]
> can
are[vbcol=seagreen]
> users
handles[vbcol=seagreen]
thousands[vbcol=seagreen]
by[vbcol=seagreen]
lead[vbcol=seagreen]
> beyond
It[vbcol=seagreen]
> East
servers[vbcol=seagreen]
better
>

How large can SQLServer scale?

I'm considering options for a large scale data warehouse. Even though SQL can theorectially scale to 10 Terabytes plus, in practice - will it be able to do it? Has anyone else actually done it? Or should Oracle be used?This is blasphemy...

Oracle

Scale out not up...

Get some serious hardware and a lot of cash...

Why 10 terabytes...

What the subject matter?|||You get into a matter of what makes sense.

1 gigabyte is trivial. MSDE will eat that alive, and ask for more.

10 gigabytes is easy. It is beyone the range of MSDE, and probably needs somebody to casually watch over it.

100 gigabytes is more challenging, you need to at least pay attention to what you are doing. A full time dba starts to make sense.

1 terabyte requires serious planning. You need to think through what you are doing, have enterprise grade hardware, and leash the bozos so that they don't make a mess of your data.

10 terabytes is where the systems I know about start to peter out. There are a few dozen of them like Terraserver (http://terraserver.microsoft.com/) and Barnes and Noble (http://www.bn.com), but they aren't all that common.

160 terrabytes is the limit that I know about, although that isn't a public server. It has a small army of DBAs and developers, but it has run for about ten months without a minute of downtime so I'd say that it is "field tested" at the very least.

-PatP|||http://www.entmag.com/news/article.asp?EditorialsID=6032|||BIG BLUE

According to Winter Corp., the largest OLTP production database is an IBM DB2 cluster system run by Land Registry. That mainframe-based system is 18.3 terabytes.

Sunday, February 19, 2012

how good is Fast Text Search

Will Fast Text Search scale and perform well on large data? Is anybody
successful with it in a big way? Who?
Thanks,
TI assume you mean, "Full Text Search"? It can be made to scale, yes--I'm
aware of some massive web sites that are using it. That said, to make it
scale in certain scenarios you need to use some tricks. I recommend reading
the following:
http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/ftslesld.mspx
Adam Machanic
SQL Server MVP
Author, "Expert SQL Server 2005 Development"
http://www.apress.com/book/bookDisplay.html?bID=10220
"Tina" <TinaMSeaburn@.nospamexcite.com> wrote in message
news:eVdtMeAuHHA.3404@.TK2MSFTNGP03.phx.gbl...
> Will Fast Text Search scale and perform well on large data? Is anybody
> successful with it in a big way? Who?
> Thanks,
> T
>|||If you do mean Fast Text Search (http://www.fastsearch.com/) and not SQL
FTS, it is extremely fast but requires an index which is at least the same
size of the data you are indexing.
--
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Tina" <TinaMSeaburn@.nospamexcite.com> wrote in message
news:eVdtMeAuHHA.3404@.TK2MSFTNGP03.phx.gbl...
> Will Fast Text Search scale and perform well on large data? Is anybody
> successful with it in a big way? Who?
> Thanks,
> T
>|||I just took a quick glance at the web site, and it looks like it doesn't
work inside of SQL Server? If that is the case, why not just use Lucene?
Adam Machanic
SQL Server MVP
Author, "Expert SQL Server 2005 Development"
http://www.apress.com/book/bookDisplay.html?bID=10220
"Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
news:%239qunrCuHHA.768@.TK2MSFTNGP04.phx.gbl...
> If you do mean Fast Text Search (http://www.fastsearch.com/) and not SQL
> FTS, it is extremely fast but requires an index which is at least the same
> size of the data you are indexing.
> --
> Hilary Cotter
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> Looking for a FAQ on Indexing Services/SQL FTS
> http://www.indexserverfaq.com
> "Tina" <TinaMSeaburn@.nospamexcite.com> wrote in message
> news:eVdtMeAuHHA.3404@.TK2MSFTNGP03.phx.gbl...
>> Will Fast Text Search scale and perform well on large data? Is anybody
>> successful with it in a big way? Who?
>> Thanks,
>> T
>|||FAST is faster than Lucene. The support for FAST is better than lucene, well
let me rephrase that, you have to pay for support for FAST, whereas with
lucene its free, and you get what you pay for.
So if you need speed (like acoona.com), you might want to use FAST. If you
want to do things on the cheap (like technorati) use lucene.
As you point out neither FAST nor lucene are integrated into SQL Server so
you have to do some work to get them to index database content.
--
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Adam Machanic" <amachanic@.IHATESPAMgmail.com> wrote in message
news:%23G7PhtCuHHA.668@.TK2MSFTNGP05.phx.gbl...
>I just took a quick glance at the web site, and it looks like it doesn't
>work inside of SQL Server? If that is the case, why not just use Lucene?
>
> --
> Adam Machanic
> SQL Server MVP
> Author, "Expert SQL Server 2005 Development"
> http://www.apress.com/book/bookDisplay.html?bID=10220
>
> "Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
> news:%239qunrCuHHA.768@.TK2MSFTNGP04.phx.gbl...
>> If you do mean Fast Text Search (http://www.fastsearch.com/) and not SQL
>> FTS, it is extremely fast but requires an index which is at least the
>> same size of the data you are indexing.
>> --
>> Hilary Cotter
>> Looking for a SQL Server replication book?
>> http://www.nwsu.com/0974973602.html
>> Looking for a FAQ on Indexing Services/SQL FTS
>> http://www.indexserverfaq.com
>> "Tina" <TinaMSeaburn@.nospamexcite.com> wrote in message
>> news:eVdtMeAuHHA.3404@.TK2MSFTNGP03.phx.gbl...
>> Will Fast Text Search scale and perform well on large data? Is anybody
>> successful with it in a big way? Who?
>> Thanks,
>> T
>>
>|||"Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
news:OFasZ8EuHHA.4476@.TK2MSFTNGP03.phx.gbl...
> FAST is faster than Lucene. The support for FAST is better than lucene,
> well
Wikipedia uses Lucene. Fast enough for the number 10 web site in the US ==probably fast enough for me :)
Adam Machanic
SQL Server MVP
Author, "Expert SQL Server 2005 Development"
http://www.apress.com/book/bookDisplay.html?bID=10220

how good is Fast Text Search

Will Fast Text Search scale and perform well on large data? Is anybody
successful with it in a big way? Who?
Thanks,
T
I assume you mean, "Full Text Search"? It can be made to scale, yes--I'm
aware of some massive web sites that are using it. That said, to make it
scale in certain scenarios you need to use some tricks. I recommend reading
the following:
http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/ftslesld.mspx

Adam Machanic
SQL Server MVP
Author, "Expert SQL Server 2005 Development"
http://www.apress.com/book/bookDisplay.html?bID=10220
"Tina" <TinaMSeaburn@.nospamexcite.com> wrote in message
news:eVdtMeAuHHA.3404@.TK2MSFTNGP03.phx.gbl...
> Will Fast Text Search scale and perform well on large data? Is anybody
> successful with it in a big way? Who?
> Thanks,
> T
>
|||If you do mean Fast Text Search (http://www.fastsearch.com/) and not SQL
FTS, it is extremely fast but requires an index which is at least the same
size of the data you are indexing.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Tina" <TinaMSeaburn@.nospamexcite.com> wrote in message
news:eVdtMeAuHHA.3404@.TK2MSFTNGP03.phx.gbl...
> Will Fast Text Search scale and perform well on large data? Is anybody
> successful with it in a big way? Who?
> Thanks,
> T
>
|||I just took a quick glance at the web site, and it looks like it doesn't
work inside of SQL Server? If that is the case, why not just use Lucene?

Adam Machanic
SQL Server MVP
Author, "Expert SQL Server 2005 Development"
http://www.apress.com/book/bookDisplay.html?bID=10220
"Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
news:%239qunrCuHHA.768@.TK2MSFTNGP04.phx.gbl...
> If you do mean Fast Text Search (http://www.fastsearch.com/) and not SQL
> FTS, it is extremely fast but requires an index which is at least the same
> size of the data you are indexing.
> --
> Hilary Cotter
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> Looking for a FAQ on Indexing Services/SQL FTS
> http://www.indexserverfaq.com
> "Tina" <TinaMSeaburn@.nospamexcite.com> wrote in message
> news:eVdtMeAuHHA.3404@.TK2MSFTNGP03.phx.gbl...
>
|||FAST is faster than Lucene. The support for FAST is better than lucene, well
let me rephrase that, you have to pay for support for FAST, whereas with
lucene its free, and you get what you pay for.
So if you need speed (like acoona.com), you might want to use FAST. If you
want to do things on the cheap (like technorati) use lucene.
As you point out neither FAST nor lucene are integrated into SQL Server so
you have to do some work to get them to index database content.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Adam Machanic" <amachanic@.IHATESPAMgmail.com> wrote in message
news:%23G7PhtCuHHA.668@.TK2MSFTNGP05.phx.gbl...
>I just took a quick glance at the web site, and it looks like it doesn't
>work inside of SQL Server? If that is the case, why not just use Lucene?
>
> --
> Adam Machanic
> SQL Server MVP
> Author, "Expert SQL Server 2005 Development"
> http://www.apress.com/book/bookDisplay.html?bID=10220
>
> "Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
> news:%239qunrCuHHA.768@.TK2MSFTNGP04.phx.gbl...
>
|||"Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
news:OFasZ8EuHHA.4476@.TK2MSFTNGP03.phx.gbl...
> FAST is faster than Lucene. The support for FAST is better than lucene,
> well
Wikipedia uses Lucene. Fast enough for the number 10 web site in the US ==
probably fast enough for me

Adam Machanic
SQL Server MVP
Author, "Expert SQL Server 2005 Development"
http://www.apress.com/book/bookDisplay.html?bID=10220

how good is Fast Text Search

Will Fast Text Search scale and perform well on large data? Is anybody
successful with it in a big way? Who?
Thanks,
TI assume you mean, "Full Text Search"? It can be made to scale, yes--I'm
aware of some massive web sites that are using it. That said, to make it
scale in certain scenarios you need to use some tricks. I recommend reading
the following:
http://www.microsoft.com/technet/pr...e/ftslesld.mspx
Adam Machanic
SQL Server MVP
Author, "Expert SQL Server 2005 Development"
http://www.apress.com/book/bookDisplay.html?bID=10220
"Tina" <TinaMSeaburn@.nospamexcite.com> wrote in message
news:eVdtMeAuHHA.3404@.TK2MSFTNGP03.phx.gbl...
> Will Fast Text Search scale and perform well on large data? Is anybody
> successful with it in a big way? Who?
> Thanks,
> T
>|||If you do mean Fast Text Search (http://www.fastsearch.com/) and not SQL
FTS, it is extremely fast but requires an index which is at least the same
size of the data you are indexing.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Tina" <TinaMSeaburn@.nospamexcite.com> wrote in message
news:eVdtMeAuHHA.3404@.TK2MSFTNGP03.phx.gbl...
> Will Fast Text Search scale and perform well on large data? Is anybody
> successful with it in a big way? Who?
> Thanks,
> T
>|||I just took a quick glance at the web site, and it looks like it doesn't
work inside of SQL Server? If that is the case, why not just use Lucene?
Adam Machanic
SQL Server MVP
Author, "Expert SQL Server 2005 Development"
http://www.apress.com/book/bookDisplay.html?bID=10220
"Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
news:%239qunrCuHHA.768@.TK2MSFTNGP04.phx.gbl...
> If you do mean Fast Text Search (http://www.fastsearch.com/) and not SQL
> FTS, it is extremely fast but requires an index which is at least the same
> size of the data you are indexing.
> --
> Hilary Cotter
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> Looking for a FAQ on Indexing Services/SQL FTS
> http://www.indexserverfaq.com
> "Tina" <TinaMSeaburn@.nospamexcite.com> wrote in message
> news:eVdtMeAuHHA.3404@.TK2MSFTNGP03.phx.gbl...
>|||FAST is faster than Lucene. The support for FAST is better than lucene, well
let me rephrase that, you have to pay for support for FAST, whereas with
lucene its free, and you get what you pay for.
So if you need speed (like acoona.com), you might want to use FAST. If you
want to do things on the cheap (like technorati) use lucene.
As you point out neither FAST nor lucene are integrated into SQL Server so
you have to do some work to get them to index database content.
--
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Adam Machanic" <amachanic@.IHATESPAMgmail.com> wrote in message
news:%23G7PhtCuHHA.668@.TK2MSFTNGP05.phx.gbl...
>I just took a quick glance at the web site, and it looks like it doesn't
>work inside of SQL Server? If that is the case, why not just use Lucene?
>
> --
> Adam Machanic
> SQL Server MVP
> Author, "Expert SQL Server 2005 Development"
> http://www.apress.com/book/bookDisplay.html?bID=10220
>
> "Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
> news:%239qunrCuHHA.768@.TK2MSFTNGP04.phx.gbl...
>|||"Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
news:OFasZ8EuHHA.4476@.TK2MSFTNGP03.phx.gbl...
> FAST is faster than Lucene. The support for FAST is better than lucene,
> well
Wikipedia uses Lucene. Fast enough for the number 10 web site in the US ==
probably fast enough for me
Adam Machanic
SQL Server MVP
Author, "Expert SQL Server 2005 Development"
http://www.apress.com/book/bookDisplay.html?bID=10220