Random Stuff

Headline today that came through one of my RSS feeds. I have to ask… Is this really news?

“Exercise helps keep weight off.”

A job has been handed to me on a silver platter. A job back at Microsoft. But this is not just any job. This is back to the same group on the same team that I worked with 12 years ago. Add to that all of the people are good friends. They are also people I work with exceptionally well as part of a team. In short it is the dream team I would have put together given a choice.

Interviews were a formality. Sallary negotiations consisted of them giving me what I told them I would take and then they added some more. But the real reason I accepted is the team I will be working with. I am being brought in as a senior member in the team. But make no mistake, these people are stars. People who I not only respect personally but professionally as well.

It was a choice that I did not take lightly. It kept me up at night thinking about this. Should I? Shouldn’t I? But, again, it all boiled down to the people.

Nope.

Ok, time for some boring stuff.

I’ve made some major improvements on the back end of my search engine. Response times so far are down to under a second for most queries. There are still some edge cases that still hang things up. I was able to make a query that took 15 minutes. But I think the most common cases are handled. Boolean searches are now possible. Scoring is a bit more accurate I hope.

I wiped the database clean and started repopulating it based on a different seed address (my own) so those that I have links to and those I have mentioned here will most likely be in there.

Right now about 5000 pages are indexed. 221,000 urls are left to be scanned. I read somewhere you should get about 8 links per page scanned, I am getting closer to 50 per page. I think that’s because I’ve scanned blogs and have managed to start indexing the BBC news site (I have a couple BBC links in my blog).

I can now run multiple crawlers across multiple machines across the internet if I so desire. I’ve limited them to 1000 sites per night and I am only running two, so I don’t piss off my gracious host (When your t1 is free, you try to be nice so you can keep it free).

Some things I plan on doing when I get the time…
1. Clean up the html
2. Create a logo
3. Write the typical “About”, “Legal”, and “Contact” pages
4. Create a page that lets people submit addresses to be indexed
5. Figure out a way to “update” the index for pages already scanned and how often to scan them
6. Write a better alogorithm for getting the abstract from the article.
7. Use the document title in the link instead of the link itself
8. Compress similar pages together (like subpages of a blog)
9. Write relevance ranking algorithm to compliment in text scoring.
10. Do more html parsing for relevance.
11. Figure out how to span multiple machines with the DB itself as it grows
12. Check out load balancing for both apache and the db. I’m sure my poor little web server wouldn’t handle a heavy load.

Maybe I should give my search site it’s own blog.
I think I might need to study up on information retrieval as well.

I’ve figured out some complicated items on search queries and how to handle them using forward indexes. I don’t have them implemented on the main page yet, but experimenting around I am now able to handle single word searches, multiple word searches and search phrases using the forward indexing.

There is still a long way to go yet. Multiple phrases and boolean expressions are not yet handled. Yet at the same time the queries now take 1/10 the time they used to.

http://rwack.com  (It also wouldn’t hurt if an ad or two was clicked )

There are also some things I need to implement which require I start over on the database so I get to wipe it out and start over.

Saying hello and introducing myself to someone from overseas in their native language.
But it’s so much fun!
Not that my ego needs to get any bigger…

This has to be one of my favorite interviews ever! Kudos to the guy being “interviewed”. The TV journalist is asking insulting questions to those waiting in line for an iPhone. He really picked the wrong guy (or right guy in my opinion) to ask “Have you ever seen a woman naked?”

My favorite part starts at the 1:34 mark where the interviewee clearly states “This is journalism to you?!” which strikes me as a pretty big insult to any journalist.

The price of gas is certainly a hot topic these days. I know personally it has hit my pocket book quite hard. A couple years ago I bought a car I never thought I would buy simply because it was getting too expensive to drive one of my gas hogs every day to work. When I did this I went from no car payment and $1200 a month for gas, down to a $575 per month car payment and a fuel bill of $300 per month. A decent amount of savings. Now that gas prices have doubled since then, I am almost back at the point I started at. For the first time since I was 20 years old, I am seriously considering riding the bus because it is getting too expensive for me to drive.

But I’m not angry about it. Far from it, really.

Over the last few years I’ve been in many discussions about fuel prices, fuel efficiency, electric cars, traffic, and a few other related topics. Usually there are a few different catalysts to the conversation or debate, but my argument as always stood pretty much the same. I usually emphasize two of my opinions. So far they seem to be playing out as expected.

People won’t change unless they are forced to do so. This is common for a lot of systems outside of people. A system does not change unless it gets stressed to the point of being either uncomfortable or even failure. People will not give up their cars unless it becomes too much of a burden to keep it. The environment, global warming, pollution, none of these have an impact on the driving habits of most people. Yes there are a few who take these things seriously, but the majority of the population will take no action regarding these impacts of a car. Why? Because they can still afford to drive.

Stress to the system will also drive innovation. The electric car has been around since the time of Henry Ford. But in the last 100 years, there has been a negligible amount of research to make electric cars go further, faster, and more reliably. Yet look at the gains in automotive technology. The worlds fastest car in 1908 could only go 141 miles per hour. It had only 135 horsepower. My “economy” car has twice the power, and with very little modification could go just as fast. But the electric car, up until recently has not really improved in performance at nearly the same pace. The power may be close, but the range was limited to about 50 miles at best. But recently, there has been real change in this area. Take a look at the Chevrolet Volt or the Tesla Roadster. Real change is coming. Why? Because people has started to demand it as a result of Gas prices. It is getting too expensive to put fuel in your car. Research is now actively happening. Making electric cars that cost less to drive and are just as convenient as gas powered cars. And they look good too.

But what is the price per gallon which people start serious making the change? The point at which it goes from only people who are early adopters to mainstream? I have always argued it will be when gas here in the U.S. goes above $5.00 per gallon. I’ve mentioned this over the course of several years as I talked to people about this subject. Recently when Honda came out with their hybrid car, and again when Toyota came out with theirs, and even yesterday when talking about the Tesla roadster. Daily we seem to get closer to that point. The new technology started to appear around $3.50 per gallon and really started to look like a serious alternative when the price went to $4.00 per gallon. Now, as we get closer to $5.00 it seems more and more like an electric car will be a reality.

I don’t know for sure. Those are just my opinions. But I think we are ready for this now.

pic 334

Originally uploaded by jesse_hires

Just another example of why things are so wonderful when the sun comes out here in Seattle

pic 419

Originally uploaded by jesse_hires

I always get a kick out of the July 4th fireworks on Lake union. This year was no different. I got what I think are some pretty spectacular pictures this time around.

I took about 400 pictures this time around of people, boats, and of course the fireworks display. Head on over to my flickr page to check them all out

Debbie and I spotted a Google StreetView camera car in my neighborhood this weekend. I wonder what the pictures will look like.