Cloud = Leaks

Story: Amazon Still Won't Talk About Government Requests For User DataTotal Replies: 12
Author Content
penguinist

Mar 25, 2015
10:07 PM EDT
This is exactly why I am so strongly opposed to the "cloud" concept. A cloud is someone else's server. If you put your data on someone else's server then you no longer control it.

For me and anyone who listens to my advice, it will be my own server that holds my data. I own the data, I own the server, and I have full control of it all.
mbaehrlxer

Mar 25, 2015
11:06 PM EDT
if you are hosting the server in a 3rd-party datacenter, you already give up control. sure, you can try to lock down the server from intruders, but it's very hard to lock down a remote machine from anyone with physical access.

if you really want to be in control of your own data, you'll need your own datacenter. without that, all you can do is damage control. encrypt disks, control what data you are actually holding (not putting any private/personal data on the server, unless it's meant to be published), purge sensitive data you no longer need, etc...

the level of control is fluent from hosting your own hardware (that you shipped to the datacenter) through root-servers to virtual servers and containers.

i tried to host my own servers at home. the maintenance and hardware cost was just to high. the internet connection alone was more expensive than getting a root-server elsewhere. so while i agree with your thoughts in principle, i just can't afford to follow them. i have to accept that i have no control over the hardware, and that someone might snap copies of my disks without me knowing it. running linux i can hopefully trust and verify that there are no trojans in the OS, but that's about it.

it's that, or i could close down my business.

greetings, eMBee.
penguinist

Mar 26, 2015
11:04 AM EDT
mbaehrlxer, I'm sorry to hear that you were unable to set up your own servers either at home or in a datacenter. Many of us however have succeeded in both.

First let's talk about a server at home:

You say that the cost of maintenance and hardware was too high. You might elaborate on that. I've had a server running in my home (and another in my vacation home) since the late 1990's. Basically, if you can afford a broadband connection and you have a spare desktop or even smaller machine laying around then you have it. I always configure the broadband connection in bridge (passthrough) mode so that the IP address is owned by my server and not by the dsl or cable modem. The rest of the setup is all just software tailoring. At my home I did invest in a high end system since I use that system for automating and storing offsite backups, but my vacation home server is nothiing more than a repurposed $150 lowend notebook (notebooks are great for this since their batteries give you the benefit of built-in UPS power).

Now let's talk about datacenter:

I understand that some people surrender their control to a data center. Some ways to do that would be to lease a virtual instance on someone else's hardware, or rent server hardware from the data center. Yes, I agree with you that neither of those choices puts you in control. What I do is lease a bare rack from a data center and I get the key to the cabinet. What I expect from the datacenter is reliable power and a big redundant pipe to the backbone. Nothing more. What I do for myself is: I build out and install my own hardware, have sole access to that hardware, do my own backups and administration. Your point about physical access being somehow insecure is only true if someone with a key to the cabinet actually reboots the system in order to compromise it. If that were to ever occur, my logs would show it. By the way, after 10 years I have exactly zero instances of a physical compromise. So, my datacenter servers are definitely under my control.

Back to your cost point, yes it is costly to lease private rack space at a data center, so that solution is only available if you have some business purpose that can support the cost, however, putting up a home server as a frontend to your broadband connection is an option that is available to any Linux user.
JaseP

Mar 26, 2015
11:38 AM EDT
I agree with mbaehrlxer... There's a big difference between a home/small office server and a data center server in terms of the horsepower required. Many organizations are right on the cusp of having big data center needs and having SMB budget. That doesn't lend itself to having a dedicated server environment, with full services. On top of that, not every ISP is going to give you a dedicated IP address. Many will just temporarily assign you one from their assigned pool (and putting the modem in pass-through mode may violate some terms of service agreements, as well, especially in a home server environment). You can even be on a sub-net of a sub-net, and not know it. That means your modem may have one "external" IP address, and your end host may see another. That translates to needing outsourcing... Whether it be hosting services or a service providing a dedicated external IP address. Either way, that means having a lack of control.

It's telling that the IT departments of some big organizations, including universities, choose to outsource services to the likes of Google, instead. These organizations have privacy requirements, many proscribed by law (HIPAA, etc.). Yet they still choose to outsource these services. It comes down to reading the service contracts very carefully, and knowing what your rights are (state & federal, as well as what they are based on what your setup is).
jdixon

Mar 26, 2015
12:04 PM EDT
> It comes down to reading the service contracts very carefully, and knowing what your rights are (state & federal, as well as what they are based on what your setup is).

It also comes down to having the legal resources to enforce those rights in court and being willing to have your information provided to law enforcement with a simple letter rather than a full warrant/subpoena.

But I agree that a small home server on broadband does not equate to a datacenter server.
mbaehrlxer

Mar 26, 2015
1:24 PM EDT
i have been running a server out of my grandmothers home for years, but in order to achieve that, i had to pay for an expensive business internet service (120 euro/month) in order to get decent upload speed and a static IP. i don't know what the cost of a root-server would have been when i started, but towards the end, a root-server was available for half that money

the maintenance cost became a problem when i no longer lived with my grandmother. i had to ask others, friends or professionals (who would charge 100/euro per hour) to go in and fix problems with the server, to the point where it just wasn't worth it, and i just let the server die.

and if i look at my work now, i am just to busy to bother dealing with hardware problems. so even if i could run a server at home, i still would not do it. at least not for services that need to remain online reliably. my workstation is acting as a home-server, and no-one cares if that goes down, so i can fix it whenever convenient.

same goes for hardware at a datacenter. had that too at one company where i worked. having to go in to fix issues that could not be dealt with remotely were a hassle. my hourly rate is many times more than what it costs to have someone else maintain the hardware for me. it's just not worth it.

greetings, eMBee.
ljmp

Mar 26, 2015
1:48 PM EDT
JaseP wrote:It's telling that the IT departments of some big organizations, including universities, choose to outsource services to the likes of Google, instead.


There are a few reasons to outsource to Google.
  1. Google rep systems prevent delivery of external email
  2. Google is `hooking` customers with the free for 5 year plan
  3. Google provides ease to configure SPF, DKIM, DMARC
Having said that, using Google services is going to come at cost - somewhere, at sometime... and it won't be pretty.

Running a server, of any kind, on a residential ISP connection violates every Term Of Service contract I've seen from any ISP I've looked at.

Having said that, I hardly believe any ISP is going to clamp down on you running a small time web server from your home for something like family calendar sharing or the like -- you most likely will not be able to send email from your home server, unless you send your data through a DYN service. However, the DYN service server rep will most likely be too low to make initial email contact with an unknown addressee. In general, it's just a bad idea to run a server from your home using a residential connection.

So, I typically just rent a virtual server - use encrypted spaces for data at rest protection - and protect the front side as best as possible - knowing the back side is already compromised by default, but usually only accessible to local attacks either intentional or unintentional.
jdixon

Mar 26, 2015
3:20 PM EDT
> Running a server, of any kind, on a residential ISP connection violates every Term Of Service contract I've seen from any ISP I've looked at.

Not if it doesn't use or provide services to the Internet it doesn't.

You can run your own file server, webmail server, streaming video server, whatever; as long as you only connect to it from within your home network. Getting and sending your mail with fetchmail and smtp uses the standard protocols and doesn't violate any agreements.

And if you connect to your machine with something like Teamviewer, then you can even access those services remotely, though I don't think Teamviewer handles video playback very well.

> So, I typically just rent a virtual server.

I've considered Amazon's free tier of service a couple of times, but I don't really have much use for it right now.
ljmp

Mar 26, 2015
3:40 PM EDT
@jdixon:

  1. Yes, of course. And I do... and many... and often... and it's great fun and learning too.
  2. Linode is very good. It's far easier to setup a few servers at linode than trying to figure out the extra tooling and terms and pricing for AWS
Of course, if you actually do need a full data center, than you've probably got the business case and cash on hand to support a full blown virtual data center. But even with Linode, running a clean static IP, you'll probably need to pay for whitelisting service through senderscore or something -- if you want to deliver email to possibly unknown users. It's expensive and a pain -- and for small time websites with a few newsletters per year, it's just easier to use Google services. Alas! The monster has won.
linux4567

Mar 31, 2015
1:18 AM EDT
@ljmp: nonsense, I have had a Linode virtual server for many years running a mail server and I never had any issues with any blacklists. Linode IPs are not blacklisted anywhere unless you do something that gets your particular IP blacklisted (such as use it to send spam).

Saying it's easier to use Google services is like saying life in prison is easier, no need to worry about taxes, bills, mortgages, insurances, finding a job or buying food, it's all provided for you as long as you don't query the quality of what you get. If you prefer to live in a prison then I guess Google is for you.

ljmp

Mar 31, 2015
10:02 AM EDT
@linux4567:

Linode runs a great shop. The IPs are usually clean. However, blacklists are only a small portion of a domain's reputation.

If you run an email server, you should definitely configure:
  1. SPF
  2. DKIM
  3. DMARC
If you haven't configured these on your server and in DNS records, it's likely that your email will get rejected from time to time when trying to send to domains like Verizon, AOL, Yahoo... Google actually accepts most email at its server, and then places it in the user's spam box if the domain rep isn't high enough.

Things become really interesting after you setup DMARC and you get reports from Google and others listing hundreds of emails each day that seemingly come from your domain. These are spoofed 'from' addresses and each one is a mark against your domain in the email reputation game.

Here's a single record from one of my domains DMARC reports from Google:

  <record>
    <row>
      <source_ip>177.143.37.188</source_ip>
      <count>1</count>
      <policy_evaluated>
        <disposition>reject</disposition>
        <dkim>fail</dkim>
        <spf>fail</spf>
      </policy_evaluated>
    </row>
    <identifiers>
      <header_from>example.com</header_from>
    </identifiers>
    <auth_results>
      <spf>
        <domain>example.com</domain>
        <result>fail</result>
      </spf>
    </auth_results>
  </record>


I've changed the 'header from' because this is a public forum. The IP listed in the source_ip field is not my server, and is not controlled by my domain. In fact, it's possible the IP address listed is someone else's email server that has been misconfigured as an open relay or compromised due to poor security practices. The above is an example of a spoofed mail message, and if I did not have DKIM, SPF, and DMARC setup - this message would have been marked as spam, sent to the user anyway, and logged as a spam mail sent from my domain --- even though it actually wasn't sent from anything I control.

In the case of DMARC reports from Microsoft's Hotmail, you can get a copy of the message that was attempted to be sent under a spoofed address. Those are always humorous, except when you remember that those messages seemingly came from your domain before you set up the domain protection protocols.

References:

Includes simplistic Gmail rep calculations (PDF):

http://www.ceas.cc/2006/19.pdf

Shows how SPF, DKIM, and DMARC work to protect domain rep (PDF):

http://www.trustedsource.org/download/research_publications/tram2007_taxonomy.pdf

Opendkim website:

http://www.opendkim.org/

SPF website and RFC:

http://www.libspf2.org/

http://tools.ietf.org/html/rfc6652

Opendmarc website:

http://www.trusteddomain.org/opendmarc.html

Verizon's Whitelist Request form:

http://my.verizon.com/micro/whitelist/RequestForm.aspx?id=isp

Note the check box for SPF. If you don't run SPF, Verizon will *not* whitelist you.
linux4567

Apr 01, 2015
1:42 PM EDT
@ljmp: I only have SPF set up for my domains and have never encountered any problems with this. All my emails have always reached the recipient. None of the big mail providers have ever refused or delayed my emails.

DKIM is definitely not necessary when using SPF and I never even heard of DMARC before.

What's more important is a clean setup with SPF, MX record and forward and reverse DNS entries for the mail server IP(s) all matching each other.

I'm well aware of spam emails that spoof my domains as sender but SPF clearly covers that as it specifies the only mail servers allowed to originate email from my domains.

I'm not interested in receiving any reports of spam email sent by spammers faking my domains, what's the point? It's not like I can do anything about it anyway.

FYI, I have been running personal mail servers for almost 20 years, and as a sysadmin maintained mail servers for a couple of large ISPs/Telcos in the past, so I do have quite some experience with mail servers.
ljmp

Apr 01, 2015
2:16 PM EDT
@linux4567:

DMARC allows you to set a policy in a DNS record that a compliant receiver should use to guide rejection/acceptance/reporting decisions. Best practices dictate that a DMARC policy record is entered even if you don't wish to receive reports. Without a DMARC policy record, email that is spoofed and fails the SPF check is still sitting in someone's spam box. This is the autospam number listed in the Google paper on email that I linked to in my above post, and counts against your domain as spam. If you run a very small personal email server, this autospam number can easily be much larger than the non-spam number.

While DKIM is certainly not needed to prevent spoofing, it does provide further assurance of the sending domain. It also affords significant benefits in assuring that the contents of the email have not changed after the envelope has been signed.

All three items are should be rather easy to implement if you are familiar with email servers, as you indicate. Take the advice or leave it... but Google, Yahoo, and other large providers highly recommend all three ---

For those who may be running Google services as a mail provider, here are instructions to setup Google's SPF, DKIM, DMARC...



It's a bit unfortunate that Google uses 2048 bit keys for its own domain, but only generates 1024 bit keys for Google services users. Or, at least that's my memory of the user key length....

Posting in this forum is limited to members of the group: [ForumMods, SITEADMINS, MEMBERS.]

Becoming a member of LXer is easy and free. Join Us!