EqualLogic and its ‘Dirty Bits’

REVISION, I didn’t quite understand this, luckily some people at the dell equallogic conference filled me in

This was a bit unintuitive to me so I thought I would put it out there. So, say you have a volume of 500 GB with thin provisioning enabled. Now, let’s say that you cloned a production 250 GB VM onto this volume. Now you have 250 GB in use and 250 GB free. Well now lets say you don’t need that VM anymore and you want a fresher copy of your production VM, so inside VSphere you right click on the VM and go delete from disk. Next, you clone your production VM onto your original 500 GB volume and BAM you just filled that volume! (And potentially put the volume into offline mode)

I thought it was permanently filling the space… but that would be ludicrous. Really it is very similar to what would happen if you didn’t quick format a thin provisioned volume. It would go ahead and use all of the space, thereby rendering the thin provisioning null and void. When you thin provision it draws a line in the sand, when you go past that line it draws another line farther out but even if you delete some files you can never go back to the previous line (this would be re thining the partition) eventually when the OS and the SAN support it, it should be possible to re thin a volume. The only way to reclaim this ‘used’ space is to move the data to another thin provisioned volume thereby starting the process over again.

What I think happened in my case is that the delete of the previous VM had not completed so when cloning the new one onto the partition it wasn’t writing to the beginning (now open) part of the volume and/or I hit a bug??

In reality this doesn’t turn out to be that big of an issue…. whew

What happened? This is the answer from EqualLogic support

The answer lies in the SCSI protocol.  As you know, our SAN is just a really big SCSI drive, for all that the server knows. It doesn’t know that the SAN is off on the network, and that the iSCSI initiator intercepts commands to/from the OS and redirects them over to us.
Unfortunately, the is no ‘delete’ command in the SCSI protocol. The server changes the File Table on one of the blocks of storage we’ve set aside for it the blocks it no longer needs, effectively deleting the data. But it never actually sends us a command ‘delete block 346135.’
Since we’re block storage, we are agnostic to the data stored on our blocks of disk space. The volume could be NTFS, or VMDK, we don’t know. So we dutifully mark any used blocks as dirty, and that’s how they stay, until the volume is deleted, or the server again writes over that block.
The server has the file table. So it knows where the blocks are, and which are used or not. When it reports that only 25GB of the 100GB volume are in use, you can believe it. When we say that 75GB of the 100GB are dirty, then you can believe us that at some point in time, data was written to all 75GB of space.
If you need to recover some of that space, then you will need to move the data off, delete the volume, and create a new, smaller volume. It’s the only way to recover dirty, yet un-used, space.

More discussions here


List of Functional Identifier Code’s

Well, assuming I didn’t mess up anything here they are

PG 100
NL 101
AC 102
AP 103
SA 104
BF 105
MH 106
MC 107
MK 108
IA 110
IF 111
PQ 112
CL 113
MR 125
DI 128
ED 130
AK 131
HU 132
CW 133
SL 135
TT 138
SL 139
LT 144
RY 146
RZ 147
IJ 148
NT 149
TN 150
TA 151
GR 152
CB 153
BC 155
NP 157
TJ 158
MP 159
SJ 160
TR 161
AS 163
ER 170
FC 175
FC 176
EP 179
AN 180
RD 185
AE 187
EC 188
AF 189
SV 190
SD 191
GT 194
LA 195
PK 196
TO 197
ME 198
TO 199
ME 200
ME 201
MJ 202
MG 203
SM 204
MN 205
MG 206
IM 210
BL 211
TM 212
MI 213
QM 214
PU 216
FG 217
AB 219
AH 220
CE 222
MQ 223
MA 224
MY 225
TU 227
MZ 240
DS 242
PN 244
ME 245
SU 248
AT 249
PV 250
CP 251
IE 252
PE 256
MG 259
MG 260
ME 261
ME 262
ME 263
MG 264
TO 265
MG 266
AD 267
PF 268
HV 269
HS 270
HB 271
LN 272
ID 273
PW 274
PI 275
HR 276
HN 277
HI 278
TE 283
IH 284
CV 285
KM 286
CO 290
RO 300
RO 301
RO 303
SO 304
SO 309
AQ 309
IO 310
SO 311
IO 312
QO 313
QO 315
SO 317
SO 319
SO 322
SO 323
SO 324
SO 325
SO 326
SO 350
AU 350
SO 352
AV 352
SO 353
AX 353
SO 354
AY 354
SO 355
AZ 355
SO 356
BA 356
SO 357
BB 357
SO 358
BD 358
SO 361
OC 362
SR 404
IR 410
TB 412
CR 414
IC 418
SR 419
CH 420
IS 421
DM 422
RL 423
SB 424
RW 426
MW 427
RU 429
RM 431
RX 432
RH 433
RJ 434
RK 435
LI 436
RV 437
EV 451
PL 452
ST 453
PB 455
EI 456
TP 460
TP 463
TP 466
TP 468
RB 470
SN 475
TP 485
TO 485
TP 486
TO 486
TP 490
TP 492
TP 494
MM 500
PH 503
CC 504
RN 511
MV 517
AO 521
MD 527
LR 536
ES 540
D4 561
D3 567
D5 568
SE 601
TS 602
EX 620
MO 650
GL 715
RF 753
RG 754
CP 805
PJ 806
IN 810
CI 811
CD 812
TF 813
GE 814
CS 815
OR 816
RP 818
JB 819
RA 820
FR 821
AA 822
LB 823
AG 824
TI 826
FR 827
DA 828
PY 829
PS 830
CT 831
SC 832
ME 833
BE 834
HP 835
RQ 836
HC 837
TD 838
PK 839
RQ 840
SP 841
NC 842
RR 843
CF 844
PA 845
IB 846
MX 847
MS 848
CF 849
PO 850
LS 851
PD 852
RI 853
DD 854
PR 855
SH 856
BS 857
SI 858
FB 859
PC 860
RC 861
SS 862
RT 863
TX 864
CA 865
SQ 866
PT 867
MT 868
RS 869
RS 870
CM 871
ME 872
CU 873
CQ 874
OG 875
OG 876
CJ 877
QG 878
QG 879
GP 880
CK 881
IG 882
DF 883
MF 884
UA 885
UB 886
CN 887
QG 888
QG 889
AM 893
DX 894
DX 895
QG 896
GC 920
GC 924
GC 925
GC 926
AI 928
OW 940
AR 943
RE 944
SW 945
AW 947
IO 980
IR 980
IM 980
IA 980
GF 990
NR 993
FT 996
FA 997
AL 998

I’m going to the first ever EqualLogic User Conference! (And my gripe list)

So I’m getting more and more jazzed to head to Dallas for the EL User Conference. I’m guessing I will learn a lot of stuff that I will wish that I had known before our install but hey that’s life. So my gripe list and conference conversational starters are for the moment this:

Why is it so hard to google documentation about EqualLogic? I understand if this top secret proprietary stuff, but all I want to do is be able to find manuals. Why is all of this stuff hidden behind the support site? No offense to Dell or their web developers but Google is very good at searching stuff… why don’t you let it do its job and you spend your time making your product better. When I go to buy a product the first thing I like to do is read as much about it as I can. I couldn’t get to a lot of the answers until after we brought the product… it could have swung the other way, we could have chosen something else. So, I can’t think of any reason to ‘hide’ this stuff. Get it out there!

Being able to download and then update the firmware from the EqualLogic box itself would be nice

So using an EqualLogic box is simplicity itself but everything else surrounding it is a lot more difficult! What I would desire is a ‘cookbook’ type of setup. So, for example if I bought a PS6000 and had future plans of buying more EqualLogic SANS (who wouldn’t) then (I think you would) would most likely recommend two 48 port 6224’s that you would then stack (we unfortunately ended up with the 24 port version which we will have to upgrade soon since we outgrew it, this sucks. I mean the whole point of EL is to be able to add, add, add to the virtual storage pool… well if I run out of ports I can’t very well do that can I?) Also, be very clear of the ramifications of total number of iSCSI connections to a pool! With just a couple of ESX servers you can easily blow that number out of the water!!! This is a great post about that BTW

of course 10 gig ethernet will help this out tremendously, but we were caught with our pants down on it once and I hope others are not!

So the cookbook would have all of the information that I would need in order to actually set this up. And for the love of Pete make it so that I can find it on the Internet…. you know using something like Google!!! It sounds like this was the way EqualLogic used to function before the acquisition so I would like to see more of this type of hardware ‘bundle’ come back into play. When your customer asks what should we buy, you can have the answers, and a setup guide to make it happen!

Hmmm I’m sure I will think of more, and don’t get me wrong, our EqualLogic is working great for us!

My Spat with GERD

So awhile back I started having pain around my xiphoid process. This was kindof strange to me because typical GERD suffers are possibly overwieght (I’m scrawny), drink alchohol or coffee (I’m LDS) or pregnant (I’m not, I’m very sure), or older. So it was sortof strange although I think there is probably a family pre-disposition to this.

One thing I did do is drink a lot of Hot chocolate. And by a lot I mean like a cup a day for many years…. So of course I followed all the suggestions. Raised my bed a few inches, stopped drinking Hot chochalte, mint, and all of the other no no’s also went to the Doc and then started up on over the counter GERD meds. (priolosec)

Well I progressed to stronger meds, prevacid I think 30 mg twice a day pretty strong regimen. And I thought I was going to be on this stuff for life.

I made those changes (raising bed, don’t eat late etc… ) but I also started drinking gatorade fairly early in the morning. This was because I can’t stand drinking plain water early in the morning, I need some sort of flavor. (typically I didn’t eat or drink until 10:30 or maybe noon) I think this early morning ‘flush’ also has helped with lessinging the GERD

Well, one weekend I was low on the drugs so I started just taking one a day. Then maybe a month later I ran out and decided to try not taking them anymore. For a couple of days I still felt discomfort (I can feel when it starts to act up) but then I was able to be off the drugs! (I don’t know if your body gets somewhat ‘addicted’ to them but I sort of suspect it a little)

I don’t know if the inflammation was finally able to settle over time but I am glad to be off them! This has been about a year now. So there you have it. More info about my GERD than you ever cared to know!

Quick tip create a file copy with a different extension

i.e. if you have a bunch of .txt files and you wanted to keep them but also have a copy with .xml

go to dos promp

go to that folder of files

copy *.txt *.xml

There you go

Get data across databases sql server

select a.col1, b.col2
from db1.dbo.tab1 a, db2.dbo.tab2 b
where a.col1 = b.col2

All you sql wizards prob know this alread 🙂

count number of files in a directory .bat file

dir “C:\FOLDER” /b/s |find /v /c “::”