Forum Settings
       
Reply To Thread

Shell scripts are evilFollow

#1 Nov 26 2011 at 10:04 PM Rating: Decent
It's Just a Flesh Wound
******
22,688 posts
That is all.
____________________________
Dear people I don't like: 凸(●´―`●)凸
#2 Nov 26 2011 at 10:54 PM Rating: Excellent
Avatar
******
29,805 posts
c:\shell - Found at ocean!

____________________________
Arch Duke Kaolian Drachensborn, lvl 95 Ranger, Unrest Server
Tech support forum | FAQ (Support) | Mobile Zam: http://m.zam.com (Premium only)
Forum Rules
#3 Nov 26 2011 at 11:26 PM Rating: Excellent
Avatar
*****
11,126 posts
Dread Lörd Kaolian wrote:
c:\shell - Found at ocean!



Screenshot
____________________________
Shaowstrike (Retired - FFXI)
91PUP/BLM 86SMN/BST 76DRK
Cooking/Fishing 100


"We don't just borrow words; on occasion, English has pursued other languages down alleyways to beat them unconscious and rifle their pockets for new vocabulary."
— James D. Nicoll
#4 Nov 27 2011 at 1:46 AM Rating: Good
:ashell













get offa ma lawn!
____________________________
gbaji wrote:
I'm smarter then you. I know how to think. I've been trained in critical thinking instead of blindly parroting what I've been told.

#5 Nov 27 2011 at 2:53 AM Rating: Excellent
*****
15,781 posts
Lrn2bash
____________________________
"I have lost my way
But I hear a tale
About a heaven in Alberta
Where they've got all hell for a basement"

#6 Nov 27 2011 at 4:21 AM Rating: Decent
It's Just a Flesh Wound
******
22,688 posts
Currently I hate solaris. I'm testing to see if the system is little endian or big endian by compiling a simple c program and then running it and nomnoming the output. And then using that output to write stuff to a header file for my actual program.

The code I'm using to capture is "out=$(./a.out)" and it works perfectly on CentOS 5.6 Linux (i386), Ubuntu 10.04.2 Linux Server (x86_64), and even Mac OS X (PPC)! But on Sun Solaris 8 (SPARC) is I get the most annoying error. "test-endian.sh: syntax error at line 21: 'out=$' unexpected"

That $ should be there. (Removing it doesn't fix the problem on solaris anyway. And causes false positives on the other systems as well.)

edit: After many trials and errors, removing the $ and putting `(./a.out)` worked for all systems. Solaris is stupid.

Edited, Nov 27th 2011 5:42am by Deadgye
____________________________
Dear people I don't like: 凸(●´―`●)凸
#7 Nov 27 2011 at 5:25 AM Rating: Good
***
3,272 posts
bash or gtfo
____________________________

(V)(;,,;)(V) Why not Zoidberg?
#8 Nov 27 2011 at 11:39 AM Rating: Excellent
Scholar
****
5,159 posts
Deadgye wrote:
edit: After many trials and errors, removing the $ and putting `(./a.out)` worked for all systems. Solaris is stupid.

To be fair, I believe that is the more common way to spawn a subshell or run a shell command. Smiley: tongue

Probably could've saved some time with

./a.out > file.txt
out=`cat file.txt`
#9 Nov 27 2011 at 11:57 AM Rating: Good
In Solaris' defense, that's exactly what `` is used for. It's how I assign process output to a variable in what little scripting I do as well.

I don't disagree that shell scripts are evil, however.
____________________________
Love,
PunkFloyd
#10 Nov 27 2011 at 3:47 PM Rating: Excellent
Avatar
******
29,805 posts
Try and make a game out of it! A shell game!
____________________________
Arch Duke Kaolian Drachensborn, lvl 95 Ranger, Unrest Server
Tech support forum | FAQ (Support) | Mobile Zam: http://m.zam.com (Premium only)
Forum Rules
#11 Nov 28 2011 at 5:17 PM Rating: Good
Encyclopedia
******
31,372 posts
Deadgye wrote:
Currently I hate solaris. I'm testing to see if the system is little endian or big endian by compiling a simple c program and then running it and nomnoming the output. And then using that output to write stuff to a header file for my actual program.

The code I'm using to capture is "out=$(./a.out)" and it works perfectly on CentOS 5.6 Linux (i386), Ubuntu 10.04.2 Linux Server (x86_64), and even Mac OS X (PPC)! But on Sun Solaris 8 (SPARC) is I get the most annoying error. "test-endian.sh: syntax error at line 21: 'out=$' unexpected"

That $ should be there. (Removing it doesn't fix the problem on solaris anyway. And causes false positives on the other systems as well.)

edit: After many trials and errors, removing the $ and putting `(./a.out)` worked for all systems. Solaris is stupid.


It has nothing to do with the operating system. What shell are you using? Solaris (and most unix flavors) will default to a c-shell environment, while most linux flavors will default to (ba)sh. When writing a script you should explicitly define the shell you're going to use (eg: #!/bin/ksh, #!/bin/csh, #!/bin/bash, etc). If you don't do this, you're just interpreting the lines in the file with whatever shell your terminal is running.

The two broad shell types are C and Bourne. C includes csh and tcsh (don't ever script with the later btw). Bourne includes sh, ksh, zsh, and bash. They have two different methods for command substitution.

C-shell uses the back quotes "`". Anything within a pair of those will be replaced with the output of the command you're running:

out=`./a.out`

Bourne also uses that syntax, but the "newer" variants (bash and ksh really), use the "$(expression)" syntax as well.:

out=$(./a.out)

This has some advantages since there's an open and close symbol, making parsing a lot less subject to silliness. Sadly, despite this you usually can't nest command substitution expressions. Although you *can* nest a backtick substitution inside a $(...) substitution. Honestly, it's cleaner to set output to variables and then use those though.


Majivo wrote:
Deadgye wrote:
edit: After many trials and errors, removing the $ and putting `(./a.out)` worked for all systems. Solaris is stupid.

To be fair, I believe that is the more common way to spawn a subshell or run a shell command. Smiley: tongue

Probably could've saved some time with

./a.out > file.txt
out=`cat file.txt`



This. Usually, if I'm not just parsing output directly (usually if it's multi-line or I want to do multiple types of parsing on it), I'll set a variable for the output file first, and then dump and read to/from that file later in the script:

#!/bin/ksh

myoutput=/tmp/dummyoutput.$$

./a.out > ${myoutput}

cat ${myoutput} | <maybe a sed expression in here> | while read x y z ; do
<do something with x>
<do something with y>
<do something with z>
done


Edited, Nov 28th 2011 4:29pm by gbaji
____________________________
King Nobby wrote:
More words please
#12 Nov 28 2011 at 7:32 PM Rating: Decent
Scholar
Avatar
***
1,089 posts
In one post I've learned more then I have in a whole semester "learning" ubuntu.
____________________________
SSubZero wrote:

MNK: "OK we're gonna go in and get those items."
WHM: "Did you have a plan?"
MNK: "Plan? I was going to walk through the front door and start punching people."
#13 Nov 29 2011 at 7:31 AM Rating: Decent
It's Just a Flesh Wound
******
22,688 posts
I missed the classes where he was going over shells scripts so I was using his examples of scripts to try and piece together my own knowledge. I didn't even know you could specify bash inside a shell script like that.

All my scripts started with #!/bin/sh, but solaris was still the only system to have a problem with me attempting to use out=$(./a.out). D: I probably would have used outputting to a file and reading from the file if I realized how much easier it was to do with scripts. Outputting to files in pure c is something I didn't want to do for what was supposed to be a simple test. :p Google also wasn't giving me the best of search results so I attempted to use what I found.

I'm really liking this class though. I wonder if I'll continue to compile all my code with -Wall -Werror even when I'm not required to.

Edited, Nov 29th 2011 8:31am by Deadgye

What's even more evil than shell scripts is virtuoso. Was a pain getting that thing set up so me and my partner could work on our project. So much of a pain that it's not done yet and it was due yesterday. <.<;

Edited, Nov 29th 2011 8:33am by Deadgye
____________________________
Dear people I don't like: 凸(●´―`●)凸
#14 Nov 29 2011 at 7:38 AM Rating: Decent
******
21,717 posts
Deadgye wrote:
I didn't even know you could specify bash inside a shell script like that.


Yes, people who know very little about the technology they're working with often find said technology difficult or evil. I grew up in a windows world and only started working with linux 8 years ago. Having been handcuffed to the handicap that is the DOS/Windows command environment and "batch" files before, shell scripting to me is the most awesome thing since sliced bread. There's a reason why thousands of sysadmins will make sure that cygwin is one of the first packages installed on a new windows server.
____________________________
R.I.P. Jessica M. 5/3/2010
This post brought to you by Carl's Jr.
gbaji wrote:
You guys keep tossing facts out there like they mean something.


#15 Nov 29 2011 at 7:46 AM Rating: Decent
It's Just a Flesh Wound
******
22,688 posts
I just think they're evil because the syntax feels unnatural to me. XD Every programming language uses words and symbols that just make sense, or are at least good enough.. even assembly! Scripts are all just like question mark here, dollar sign there, quotatio backtick over yonder, etc.

They're definitely very useful and awesome.. but being awesome doesn't mean you can't also be evil.

Edited, Nov 29th 2011 8:47am by Deadgye
____________________________
Dear people I don't like: 凸(●´―`●)凸
#16 Nov 29 2011 at 9:12 AM Rating: Decent
Scholar
Avatar
***
1,089 posts
Deadgye wrote:
I just think they're evil because the syntax feels unnatural to me. XD Every programming language uses words and symbols that just make sense, or are at least good enough.. even assembly! Scripts are all just like question mark here, dollar sign there, quotatio backtick over yonder, etc.

They're definitely very useful and awesome.. but being awesome doesn't mean you can't also be evil.

Edited, Nov 29th 2011 8:47am by Deadgye


My prof's answer for this is "oh it's not working? try changing all your ` to ".
____________________________
SSubZero wrote:

MNK: "OK we're gonna go in and get those items."
WHM: "Did you have a plan?"
MNK: "Plan? I was going to walk through the front door and start punching people."
#17 Nov 29 2011 at 11:22 AM Rating: Good
***
3,272 posts
Deadgye wrote:
I just think they're evil because the syntax feels unnatural to me. XD Every programming language uses words and symbols that just make sense, or are at least good enough.. even assembly! Scripts are all just like question mark here, dollar sign there, quotatio backtick over yonder, etc.

They're definitely very useful and awesome.. but being awesome doesn't mean you can't also be evil.

Edited, Nov 29th 2011 8:47am by Deadgye


In my linux class I was spoiled with lots of script work so when I had to add users into my server environment in 2k8r2 I was pretty pissed. We'd been given no instruction on powershell or making a batch file to add multiple users so doing it by hand just seemed tedious. In my linux class when I added users all I had to do was run the script I made and shizzam, 15 users/permissions added because I can copy pasta the same line and change the name.
____________________________

(V)(;,,;)(V) Why not Zoidberg?
#18 Nov 29 2011 at 1:46 PM Rating: Good
Encyclopedia
******
31,372 posts
Deadgye wrote:
I missed the classes where he was going over shells scripts so I was using his examples of scripts to try and piece together my own knowledge. I didn't even know you could specify bash inside a shell script like that.


Yup. Personally, I prefer ksh because like sh, it's stable but has additional features, but unlike bash and tcsh it's not also a user command line shell. Shells used for command line work tend to change and add/remove features over time, which can make shell scripts break.

Quote:
All my scripts started with #!/bin/sh, but solaris was still the only system to have a problem with me attempting to use out=$(./a.out).


That's not a problem with Solaris. It's a problem with linux (and one of the many things I berate the linux development community about). Try this on your computer:

 
ls -l /bin/sh 
lrwxrwxrwx 1 root root 4 Feb 22  2011 /bin/sh -> bash* 


You're not actually running sh. You're running bash. Something which apparently the idiots who develop linux thought they'd do because "bash is better". It's stupid because bash *isn't* better if you're actually doing single user scripting/repair. There are a couple reasons for this. First is the issue with potential inconsistencies over time with a user shell like bash. Hasn't happened yet (that I know of), but I saw this first hand with tcsh about 10 years ago so it's really just a matter of time. Difference being that *nobody* used tcsh for system level scripts (like the stuff in /etc/init.d/). Um, and also someone who thinks they're writing a sh script and thinks that certain commands will work, and is then surprised when this supposedly portable shell script they wrote doesn't work on another system (like what happened to you). Don't get me started on vi linked to vim either btw.

The second reason is that ideally (and to be fair Solaris doesn't still do this) you want the default shell used by your startup scripts (and single user mode) to be a separate and statically linked binary from the one(s) used during normal operation. This is so that just in case say your /usr directory structure (which could even be on a separate disk partition) should not be available on boot, you can still boot into single user mode and fix things. It's a rare problem to run into these days, so it's not a huge huge deal, but it's still a bad thing.

Then there's this:

 
ldd /bin/sh 
        linux-vdso.so.1 =>  (0x00007fff31dfc000) 
        libtermcap.so.2 => /lib64/libtermcap.so.2 (0x00000033fa800000) 
        libdl.so.2 => /lib64/libdl.so.2 (0x00000033f7400000) 
        libc.so.6 => /lib64/libc.so.6 (0x00000033f7000000) 
        /lib64/ld-linux-x86-64.so.2 (0x00000033f6800000) 


See that first line? Can anyone in the class tell me why this is a potential security problem? Remember, that this is the shell that's run as root on startup and will initialize your network, your firewall rules, your login environment, and every other thing you computer does, or that any user does with the computer.

Linux does many things right, but they managed to screw up a few things as well.

Quote:
I probably would have used outputting to a file and reading from the file if I realized how much easier it was to do with scripts. Outputting to files in pure c is something I didn't want to do for what was supposed to be a simple test. :p Google also wasn't giving me the best of search results so I attempted to use what I found.


Honestly, it's usually easier to just pipe the output directly into whatever you want to do with it. Only dump the output to a file if you have to. Sometimes, it's the easiest way to deal with some bulky data. Doubly so if you're going to read it multiple times. A common scripting mistake is to run the same command several times parsing for different bits of output each time. If you're doing that, dump it to a file and read the file (don't forget to delete the file later though. Cleanliness is next to something important I'm sure!). But if you're just running a command and testing the output to check for something (like in your case), I'd just pipe the output directly to a sed/awk expression designed to find the bit I need and then test it.

Edited, Nov 29th 2011 11:48am by gbaji
____________________________
King Nobby wrote:
More words please
#19 Nov 30 2011 at 12:27 AM Rating: Decent
It's Just a Flesh Wound
******
22,688 posts
I'm sshing into the systems to program/compile on them so I doubt I'll have root access. (Or did that long command not need it?) This next/final hw we're using autoconf, automake, and libtool to port our program so hopefully they'll be less silly script problems. If there are though, I'll know who to ask lol.
____________________________
Dear people I don't like: 凸(●´―`●)凸
#20 Nov 30 2011 at 3:24 PM Rating: Decent
Encyclopedia
******
31,372 posts
Deadgye wrote:
I'm sshing into the systems to program/compile on them so I doubt I'll have root access.


Neither of the commands require root access. The first is just a list showing that /bin/sh is actually just a symbolic link to /bin/bash. Which is why syntax which appears to work in sh on linux will not work on solaris. The linux script is actually running bash.

The security element to this is that even though *you* may not be running as root, the root user uses that shell to run all the startup stuff. You really want to use a more stable and less subject to change shell for that. The linux devs did that linking out of sheer laziness. A lot of the devs just wrote their init script in bash. When time came to consolidate a bunch of different pieces into full distributions, rather than spend the time correctly re-writing the script to use a strict sh syntax, they went the easy route and just linked sh to bash. That way any init script written in either shell would work on any linux system. It's a great example of the exact wrong way to fix a problem.

Oh. The second command lists the dynamically loaded libraries used by a binary. Since it looks like you're doing some compiling you might find it useful. It allows you to work backwards with a binary and learn what libraries it's dependent on. Incredibly useful command for debugging odd problems with code portability btw. The security angle there is that when you dynamically link a library, the binary will behave differently depending on the libraries themselves. Normally that's necessary for portability. It means you don't have to recompile your binaries for every os/kernel/whatever it runs on. The problem is that if someone replaces a library with something malicious, bad things can happen. Usually, if the libraries are all pointing explicitly to local libraries (like /usr, /usr/local, etc), you're ok. But I've seen some pretty "interesting" library paths. It's something to think about when compiling code. Be aware of what paths you're using for libraries if you explicitly link them into your code.
____________________________
King Nobby wrote:
More words please
#21 Dec 01 2011 at 9:18 AM Rating: Decent
It's Just a Flesh Wound
******
22,688 posts
That's definitely a better alternative than going through the source code slowly and looking at all the includes everywhere. Only wish I knew about it before I ported this crappy skey package to those 4 systems lol.
____________________________
Dear people I don't like: 凸(●´―`●)凸
Reply To Thread

Colors Smileys Quote OriginalQuote Checked Help

 

Recent Visitors: 78 All times are in CDT
Kavekk, Poldaran, someproteinguy, Xsarus, Anonymous Guests (74)